Aug 04 18:27:36.559996 ip-10-0-128-139 systemd[1]: Starting Kubernetes Kubelet...
Aug 04 18:27:37.010797 ip-10-0-128-139 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 18:27:37.010797 ip-10-0-128-139 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 04 18:27:37.010797 ip-10-0-128-139 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 18:27:37.010797 ip-10-0-128-139 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 04 18:27:37.010797 ip-10-0-128-139 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 18:27:37.011841 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.011757    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 04 18:27:37.016436 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016422    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:27:37.016436 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016437    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016440    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016445    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016449    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016452    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016455    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016458    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016461    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016464    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016466    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016470    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016472    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016475    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016478    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016481    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016484    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016487    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016490    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016494    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016502    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:27:37.016502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016506    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016508    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016512    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016515    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016518    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016520    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016523    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016525    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016528    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016531    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016533    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016536    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016538    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016542    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016545    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016549    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016551    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016555    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016558    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016561    2578 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:27:37.017007 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016564    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016568    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016570    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016573    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016576    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016578    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016581    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016583    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016586    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016588    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016591    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016593    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016596    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016598    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016603    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016606    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016609    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016611    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016614    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016616    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:27:37.017490 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016619    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016622    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016624    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016627    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016629    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016632    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016634    2578 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016636    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016639    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016641    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016644    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016647    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016650    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016652    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016655    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016658    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016661    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016664    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016667    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016669    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:27:37.017992 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016673    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016675    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016678    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016680    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016683    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016686    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016689    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016691    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016694    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016698    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016714    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016718    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016721    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016724    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016727    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016731    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016733    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.016736    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017419    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:27:37.018484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017424    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017427    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017430    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017433    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017436    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017439    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017441    2578 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017444    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017446    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017449    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017452    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017456    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017458    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017461    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017464    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017467    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017469    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017472    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017474    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:27:37.019046 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017477    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017481    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017485    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017488    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017491    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017494    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017497    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017500    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017503    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017505    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017508    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017511    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017514    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017517    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017520    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017522    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017525    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017527    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017530    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:27:37.019525 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017535    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017539    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017542    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017545    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017548    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017551    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017553    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017556    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017559    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017561    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017564    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017566    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017569    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017571    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017574    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017577    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017579    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017582    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017585    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017587    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:27:37.020017 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017589    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017592    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017594    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017597    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017599    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017602    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017605    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017607    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017611    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017615    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017617    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017620    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017623    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017625    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017628    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017630    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017633    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017636    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017639    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017641    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:27:37.020502 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017644    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017646    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017649    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017651    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017654    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017656    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017658    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017661    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017664    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017666    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017669    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017671    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017674    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017676    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017679    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017681    2578 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017684    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017689    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017691    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:27:37.021028 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.017694    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018052    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018056    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018059    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018062    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018066    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018069    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018072    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018075    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018078    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018081    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018084    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018086    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018089    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018093    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018096    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018099    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018102    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018106    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:27:37.021484 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018111    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018117    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018120    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018123    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018125    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018128    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018131    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018134    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018136    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018139    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018141    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018145    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018147    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018150    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018152    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018155    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018157    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018161    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018164    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:27:37.021984 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018168    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018170    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018173    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018176    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018179    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018182    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018184    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018187    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018189    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018192    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018195    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018205    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018207    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018210    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018214    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018216    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018219    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018221    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018224    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018226    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:27:37.022452 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018229    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018231    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018234    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018236    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018241    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018244    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018246    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018249    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018251    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018254    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018256    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018259    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018261    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018267    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018270    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018273    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018276    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018278    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018281    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:27:37.022953 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018283    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018285    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018288    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018291    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018293    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018296    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018298    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018303    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018305    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018307    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018310    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018312    2578 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018315    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018317    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018320    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018322    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018325    2578 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018329    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018331    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018334    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018336    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:27:37.023432 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018339    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.018341    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018410    2578 flags.go:64] FLAG: --address="0.0.0.0"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018417    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018422    2578 flags.go:64] FLAG: --anonymous-auth="true"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018427    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018434    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018437    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018441    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018445    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018448    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018452    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018455    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018459    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018462    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018465    2578 flags.go:64] FLAG: --cgroup-root=""
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018467    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018470    2578 flags.go:64] FLAG: --client-ca-file=""
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018474    2578 flags.go:64] FLAG: --cloud-provider="external"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018477    2578 flags.go:64] FLAG: --cluster-dns="[]"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018485    2578 flags.go:64] FLAG: --cluster-domain=""
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018488    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018491    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018494    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 04 18:27:37.023957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018497    2578 flags.go:64] FLAG: --container-log-max-files="5"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018501    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018504    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018507    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018511    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018514    2578 flags.go:64] FLAG: --contention-profiling="false"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018517    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018520    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018523    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018526    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018530    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018533    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018536    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018539    2578 flags.go:64] FLAG: --enable-load-reader="false"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018542    2578 flags.go:64] FLAG: --enable-server="true"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018545    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018549    2578 flags.go:64] FLAG: --event-burst="100"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018553    2578 flags.go:64] FLAG: --event-qps="50"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018555    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018558    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018561    2578 flags.go:64] FLAG: --eviction-hard=""
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018565    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018568    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018571    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018573    2578 flags.go:64] FLAG: --eviction-soft=""
Aug 04 18:27:37.024534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018576    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018580    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018583    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018586    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018589    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018592    2578 flags.go:64] FLAG: --fail-swap-on="true"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018594    2578 flags.go:64] FLAG: --feature-gates=""
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018598    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018601    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018605    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018608    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018611    2578 flags.go:64] FLAG: --healthz-port="10248"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018614    2578 flags.go:64] FLAG: --help="false"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018616    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018619    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018622    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018625    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018628    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018631    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018634    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018637    2578 flags.go:64] FLAG: --image-service-endpoint=""
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018640    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018643    2578 flags.go:64] FLAG: --kube-api-burst="100"
Aug 04 18:27:37.025210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018646    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018649    2578 flags.go:64] FLAG: --kube-api-qps="50"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018652    2578 flags.go:64] FLAG: --kube-reserved=""
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018654    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018657    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018660    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018663    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018666    2578 flags.go:64] FLAG: --lock-file=""
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018669    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018671    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018675    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018680    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018683    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018686    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018689    2578 flags.go:64] FLAG: --logging-format="text"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018691    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018695    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018697    2578 flags.go:64] FLAG: --manifest-url=""
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018712    2578 flags.go:64] FLAG: --manifest-url-header=""
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018716    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018719    2578 flags.go:64] FLAG: --max-open-files="1000000"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018724    2578 flags.go:64] FLAG: --max-pods="110"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018727    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018730    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018733    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 04 18:27:37.025852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018735    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018738    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018741    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018744    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018750    2578 flags.go:64] FLAG: --node-status-max-images="50"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018753    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018756    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018759    2578 flags.go:64] FLAG: --pod-cidr=""
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018762    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018768    2578 flags.go:64] FLAG: --pod-manifest-path=""
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018771    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018774    2578 flags.go:64] FLAG: --pods-per-core="0"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018777    2578 flags.go:64] FLAG: --port="10250"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018780    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018783    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06ca4be79e6b6a9a5"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018786    2578 flags.go:64] FLAG: --qos-reserved=""
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018788    2578 flags.go:64] FLAG: --read-only-port="10255"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018791    2578 flags.go:64] FLAG: --register-node="true"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018794    2578 flags.go:64] FLAG: --register-with-taints=""
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018802    2578 flags.go:64] FLAG: --registry-burst="10"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018805    2578 flags.go:64] FLAG: --registry-qps="5"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018808    2578 flags.go:64] FLAG: --reserved-cpus=""
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018810    2578 flags.go:64] FLAG: --reserved-memory=""
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018814    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018817    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 04 18:27:37.026467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018820    2578 flags.go:64] FLAG: --rotate-certificates="false"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018823    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018825    2578 flags.go:64] FLAG: --runonce="false"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018828    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018831    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018834    2578 flags.go:64] FLAG: --seccomp-default="false"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018837    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018840    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018842    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018846    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018849    2578 flags.go:64] FLAG: --storage-driver-password="root"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018851    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018854    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018857    2578 flags.go:64] FLAG: --storage-driver-user="root"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018860    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018864    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018869    2578 flags.go:64] FLAG: --system-cgroups=""
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018872    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018877    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018880    2578 flags.go:64] FLAG: --tls-cert-file=""
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018883    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018888    2578 flags.go:64] FLAG: --tls-min-version=""
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018890    2578 flags.go:64] FLAG: --tls-private-key-file=""
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018893    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018896    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 04 18:27:37.027098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018899    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018903    2578 flags.go:64] FLAG: --v="2"
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018906    2578 flags.go:64] FLAG: --version="false"
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018910    2578 flags.go:64] FLAG: --vmodule=""
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018914    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.018918    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019027    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019031    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019034    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019037    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019040    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019043    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019046    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019049    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019052    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019055    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019058    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019060    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019063    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019065    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:27:37.027717 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019069    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019073    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019077    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019080    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019084    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019087    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019090    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019093    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019096    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019098    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019101    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019104    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019107    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019109    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019112    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019114    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019117    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019120    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019122    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:27:37.028212 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019125    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019127    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019129    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019132    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019134    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019137    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019139    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019142    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019145    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019147    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019149    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019152    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019154    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019157    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019160    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019163    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019166    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019168    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019171    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019174    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:27:37.028907 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019177    2578 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019179    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019182    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019184    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019189    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019191    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019194    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019196    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019198    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019201    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019203    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019206    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019208    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019211    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019213    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019216    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019218    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019221    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019223    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:27:37.029680 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019226    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019228    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019231    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019233    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019236    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019238    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019241    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019244    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019247    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019250    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019252    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019255    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019258    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019260    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019263    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019265    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019268    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019271    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019274    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019276    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:27:37.030177 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019280    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019283    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019286    2578 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019289    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019291    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019293    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.019296    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.019953    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.026619    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.026633    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.026652    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.026658    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026727    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026732    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026736    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:27:37.030681 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026740    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026743    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026746    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026749    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026752    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026755    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026758    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026761    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026764    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026767    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026769    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026772    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026775    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026777    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026780    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026783    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026785    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026788    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026792    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026795    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:27:37.031094 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026798    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026800    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026803    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026806    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026808    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026811    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026815    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026818    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026821    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026823    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026828    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026832    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026835    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026838    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026841    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026843    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026846    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026849    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026852    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:27:37.031584 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026854    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026857    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026859    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026862    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026864    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026867    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026869    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026872    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026874    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026877    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026879    2578 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026882    2578 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026884    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026887    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026889    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026892    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026894    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026897    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026899    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026902    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:27:37.032074 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026906    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026909    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026911    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026914    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026917    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026920    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026923    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026925    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026928    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026931    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026933    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026936    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026938    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026941    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026944    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026946    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026949    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026951    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026954    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026957    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:27:37.032607 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026959    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026962    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026964    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026967    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026970    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026972    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026976    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026980    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026983    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026986    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026988    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026992    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026995    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.026998    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027000    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027003    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:27:37.033115 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027006    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.027011    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027118    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027123    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027126    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027129    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027131    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027134    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027137    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027139    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027141    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027144    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027148    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027152    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 18:27:37.033497 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027155    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027158    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027160    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027163    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027165    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027168    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027170    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027173    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027176    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027178    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027181    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027183    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027186    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027188    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027191    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027194    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027197    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027200    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027202    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027205    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 18:27:37.033988 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027207    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027209    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027212    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027214    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027217    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027219    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027222    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027224    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027227    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027229    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027231    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027234    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027237    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027239    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027242    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027244    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027247    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027250    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027253    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027256    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 18:27:37.034471 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027258    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027261    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027264    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027266    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027269    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027271    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027273    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027276    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027279    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027282    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027284    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027287    2578 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027289    2578 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027292    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027295    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027297    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027299    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027302    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027304    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027307    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 18:27:37.034963 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027309    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027312    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027314    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027316    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027319    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027321    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027324    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027326    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027329    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027331    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027334    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027336    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027339    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027341    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027344    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027346    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027350    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027353    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027356    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 18:27:37.035455 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027359    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027362    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027365    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027368    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027371    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027374    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027376    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:37.027379    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.027384    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.027500    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.031202    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.032679    2578 server.go:1019] "Starting client certificate rotation"
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.032776    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 18:27:37.035964 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.032812    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 18:27:37.036374 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.035411    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 18:27:37.036511 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.036495    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 18:27:37.050757 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.050719    2578 log.go:25] "Validated CRI v1 runtime API"
Aug 04 18:27:37.053152 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.053136    2578 log.go:25] "Validated CRI v1 image API"
Aug 04 18:27:37.053937 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.053924    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 04 18:27:37.058976 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.058954    2578 fs.go:135] Filesystem UUIDs: map[1569f4d9-0619-4ce3-b44f-91e12b6be234:/dev/nvme0n1p3 40ddcfe5-916f-42a1-8885-2d894d5b9bd2:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Aug 04 18:27:37.059056 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.058975    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 04 18:27:37.061539 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.061522    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 18:27:37.063898 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.063796    2578 manager.go:216] Machine: {Timestamp:2026-08-04 18:27:37.062527877 +0000 UTC m=+0.375315220 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098497 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2998c906d6fd4bdd4a8e2547892862 SystemUUID:ec2998c9-06d6-fd4b-dd4a-8e2547892862 BootID:a746d43e-833c-410c-a128-e28c94d80b4e Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9f:7f:2f:91:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9f:7f:2f:91:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:f1:12:c1:d2:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 04 18:27:37.063898 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.063891    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 04 18:27:37.064028 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.063964    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 04 18:27:37.067226 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.067204    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 04 18:27:37.067364 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.067228    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-139.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 04 18:27:37.068108 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.068098    2578 topology_manager.go:138] "Creating topology manager with none policy"
Aug 04 18:27:37.068139 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.068110    2578 container_manager_linux.go:309] "Creating device plugin manager"
Aug 04 18:27:37.068139 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.068117    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 18:27:37.068789 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.068778    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 18:27:37.070089 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.070079    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 04 18:27:37.074888 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.074877    2578 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 18:27:37.074994 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.074985    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 04 18:27:37.075033 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.075013    2578 kubelet.go:486] "Attempting to sync node with API server"
Aug 04 18:27:37.075033 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.075023    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 04 18:27:37.075092 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.075041    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 04 18:27:37.075092 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.075051    2578 kubelet.go:398] "Adding apiserver pod source"
Aug 04 18:27:37.075092 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.075078    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 04 18:27:37.075422 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.075411    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 18:27:37.075472 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.075428    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 18:27:37.079211 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.079191    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 04 18:27:37.080939 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.080925    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 04 18:27:37.081013 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.080955    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 04 18:27:37.082293 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082281    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 04 18:27:37.082347 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082298    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 04 18:27:37.082347 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082304    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 04 18:27:37.082347 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082310    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 04 18:27:37.082347 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082322    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 04 18:27:37.082347 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082331    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 04 18:27:37.082347 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082339    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 04 18:27:37.082347 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082344    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 04 18:27:37.082347 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082350    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 04 18:27:37.082572 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082356    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 04 18:27:37.082572 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082364    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 04 18:27:37.082572 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082369    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 04 18:27:37.082572 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082392    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 04 18:27:37.082572 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.082397    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 04 18:27:37.085623 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.085606    2578 server.go:1268] "Started kubelet"
Aug 04 18:27:37.085740 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.085696    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 04 18:27:37.085740 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.085687    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 04 18:27:37.085855 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.085771    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 04 18:27:37.086466 ip-10-0-128-139 systemd[1]: Started Kubernetes Kubelet.
Aug 04 18:27:37.086740 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.086716    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 04 18:27:37.086828 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.086740    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-139.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 04 18:27:37.087379 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.087363    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 04 18:27:37.087631 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.087603    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-139.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 04 18:27:37.087631 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.087624    2578 server.go:310] "Adding debug handlers to kubelet server"
Aug 04 18:27:37.090695 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.090667    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 04 18:27:37.094330 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.093264    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-139.ec2.internal.18c8ad89364d85e1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-139.ec2.internal,UID:ip-10-0-128-139.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-139.ec2.internal,},FirstTimestamp:2026-08-04 18:27:37.085576673 +0000 UTC m=+0.398364016,LastTimestamp:2026-08-04 18:27:37.085576673 +0000 UTC m=+0.398364016,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-139.ec2.internal,}"
Aug 04 18:27:37.096080 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.096059    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 04 18:27:37.096165 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.096080    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 04 18:27:37.096233 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.096204    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-139.ec2.internal\" not found"
Aug 04 18:27:37.096835 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.096814    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 04 18:27:37.096835 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.096825    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 04 18:27:37.096963 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.096840    2578 factory.go:55] Registering systemd factory
Aug 04 18:27:37.096963 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.096858    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 04 18:27:37.096963 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.096872    2578 factory.go:223] Registration of the systemd container factory successfully
Aug 04 18:27:37.097104 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.096992    2578 reconstruct.go:97] "Volume reconstruction finished"
Aug 04 18:27:37.097104 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.097000    2578 reconciler.go:29] "Reconciler: start to sync state"
Aug 04 18:27:37.097205 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.097134    2578 factory.go:153] Registering CRI-O factory
Aug 04 18:27:37.097205 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.097148    2578 factory.go:223] Registration of the crio container factory successfully
Aug 04 18:27:37.097301 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.097211    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 04 18:27:37.097301 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.097236    2578 factory.go:103] Registering Raw factory
Aug 04 18:27:37.097301 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.097252    2578 manager.go:1196] Started watching for new ooms in manager
Aug 04 18:27:37.097799 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.097784    2578 manager.go:349] Starting recovery of all containers
Aug 04 18:27:37.100007 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.099963    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-139.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 04 18:27:37.100007 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.099980    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 04 18:27:37.112997 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.112977    2578 manager.go:354] Recovery completed
Aug 04 18:27:37.113921 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.113896    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-48c58"
Aug 04 18:27:37.114497 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.114473    2578 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Aug 04 18:27:37.117527 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.117515    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:27:37.119444 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.119426    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:27:37.119510 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.119457    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:27:37.119510 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.119468    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:27:37.119902 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.119843    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-48c58"
Aug 04 18:27:37.120173 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.120160    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 04 18:27:37.120234 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.120171    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 04 18:27:37.120234 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.120194    2578 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 18:27:37.120234 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.120175    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:27:37.120974 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.120962    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:27:37.121048 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.120985    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:27:37.121048 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.120994    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:27:37.121462 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.121393    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-139.ec2.internal.18c8ad8938524fbe  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-139.ec2.internal,UID:ip-10-0-128-139.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-139.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-139.ec2.internal,},FirstTimestamp:2026-08-04 18:27:37.119444926 +0000 UTC m=+0.432232269,LastTimestamp:2026-08-04 18:27:37.119444926 +0000 UTC m=+0.432232269,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-139.ec2.internal,}"
Aug 04 18:27:37.123254 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.123241    2578 policy_none.go:49] "None policy: Start"
Aug 04 18:27:37.123318 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.123259    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 04 18:27:37.123318 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.123274    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 04 18:27:37.125169 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.125156    2578 policy_none.go:47] "Start"
Aug 04 18:27:37.130658 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.130569    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-139.ec2.internal.18c8ad8938529257  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-139.ec2.internal,UID:ip-10-0-128-139.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-139.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-139.ec2.internal,},FirstTimestamp:2026-08-04 18:27:37.119461975 +0000 UTC m=+0.432249318,LastTimestamp:2026-08-04 18:27:37.119461975 +0000 UTC m=+0.432249318,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-139.ec2.internal,}"
Aug 04 18:27:37.142846 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.142783    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-139.ec2.internal.18c8ad893852c16b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-139.ec2.internal,UID:ip-10-0-128-139.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-139.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-139.ec2.internal,},FirstTimestamp:2026-08-04 18:27:37.119474027 +0000 UTC m=+0.432261372,LastTimestamp:2026-08-04 18:27:37.119474027 +0000 UTC m=+0.432261372,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-139.ec2.internal,}"
Aug 04 18:27:37.155028 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.154941    2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-139.ec2.internal.18c8ad8938524fbe\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-139.ec2.internal.18c8ad8938524fbe  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-139.ec2.internal,UID:ip-10-0-128-139.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-139.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-139.ec2.internal,},FirstTimestamp:2026-08-04 18:27:37.119444926 +0000 UTC m=+0.432232269,LastTimestamp:2026-08-04 18:27:37.120973845 +0000 UTC m=+0.433761189,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-139.ec2.internal,}"
Aug 04 18:27:37.163682 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.163621    2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-139.ec2.internal.18c8ad8938529257\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-139.ec2.internal.18c8ad8938529257  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-139.ec2.internal,UID:ip-10-0-128-139.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-139.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-139.ec2.internal,},FirstTimestamp:2026-08-04 18:27:37.119461975 +0000 UTC m=+0.432249318,LastTimestamp:2026-08-04 18:27:37.120989513 +0000 UTC m=+0.433776856,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-139.ec2.internal,}"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.167252    2578 manager.go:337] "Starting Device Plugin manager"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.167285    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.167294    2578 server.go:87] "Starting device plugin registration server"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.167541    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.167552    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.167680    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.167761    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.167768    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.168243    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.168286    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-139.ec2.internal\" not found"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.174663    2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-139.ec2.internal.18c8ad893852c16b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-139.ec2.internal.18c8ad893852c16b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-139.ec2.internal,UID:ip-10-0-128-139.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-139.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-139.ec2.internal,},FirstTimestamp:2026-08-04 18:27:37.119474027 +0000 UTC m=+0.432261372,LastTimestamp:2026-08-04 18:27:37.120997874 +0000 UTC m=+0.433785218,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-139.ec2.internal,}"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.186957    2578 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2612 into the system container \"/system.slice\": <nil>"
Aug 04 18:27:37.262290 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.193030    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-139.ec2.internal.18c8ad893c0e36ef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-139.ec2.internal,UID:ip-10-0-128-139.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-128-139.ec2.internal,},FirstTimestamp:2026-08-04 18:27:37.182090991 +0000 UTC m=+0.494878322,LastTimestamp:2026-08-04 18:27:37.182090991 +0000 UTC m=+0.494878322,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-139.ec2.internal,}"
Aug 04 18:27:37.268348 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.268332    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:27:37.269275 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.269260    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:27:37.269340 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.269292    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:27:37.269340 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.269305    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:27:37.269340 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.269329    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.291836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.291811    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.291913 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.291837    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-139.ec2.internal\": node \"ip-10-0-128-139.ec2.internal\" not found"
Aug 04 18:27:37.309021 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.309001    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-139.ec2.internal\" not found"
Aug 04 18:27:37.348614 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.348589    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 04 18:27:37.349695 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.349681    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 04 18:27:37.349783 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.349733    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 04 18:27:37.349783 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.349757    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 04 18:27:37.349862 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.349794    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 04 18:27:37.352237 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.352220    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:27:37.409860 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.409838    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-139.ec2.internal\" not found"
Aug 04 18:27:37.450007 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.449983    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-139.ec2.internal"]
Aug 04 18:27:37.450130 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.450063    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:27:37.453779 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.453764    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:27:37.453862 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.453791    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:27:37.453862 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.453802    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:27:37.456068 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.456056    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:27:37.456211 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.456198    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.456248 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.456232    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:27:37.457269 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.457252    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:27:37.457355 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.457284    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:27:37.457355 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.457295    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:27:37.457355 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.457253    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:27:37.457444 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.457366    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:27:37.457444 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.457378    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:27:37.459553 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.459540    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.459614 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.459577    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 18:27:37.460648 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.460629    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 18:27:37.460742 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.460655    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 18:27:37.460742 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.460664    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeHasSufficientPID"
Aug 04 18:27:37.495413 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.495395    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-139.ec2.internal\" not found" node="ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.497845 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.497825    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d0a2e4e61221718b5319001dcdbfbbc-config\") pod \"kube-apiserver-proxy-ip-10-0-128-139.ec2.internal\" (UID: \"7d0a2e4e61221718b5319001dcdbfbbc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.497914 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.497860    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a3e3b8db9dd2cfc3e4727a04a3b5eb0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal\" (UID: \"a3e3b8db9dd2cfc3e4727a04a3b5eb0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.497955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.497916    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3e3b8db9dd2cfc3e4727a04a3b5eb0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal\" (UID: \"a3e3b8db9dd2cfc3e4727a04a3b5eb0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.499802 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.499788    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-139.ec2.internal\" not found" node="ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.510278 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.510260    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-139.ec2.internal\" not found"
Aug 04 18:27:37.598814 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.598752    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d0a2e4e61221718b5319001dcdbfbbc-config\") pod \"kube-apiserver-proxy-ip-10-0-128-139.ec2.internal\" (UID: \"7d0a2e4e61221718b5319001dcdbfbbc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.598814 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.598783    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a3e3b8db9dd2cfc3e4727a04a3b5eb0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal\" (UID: \"a3e3b8db9dd2cfc3e4727a04a3b5eb0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.598814 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.598801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3e3b8db9dd2cfc3e4727a04a3b5eb0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal\" (UID: \"a3e3b8db9dd2cfc3e4727a04a3b5eb0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.598990 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.598845    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d0a2e4e61221718b5319001dcdbfbbc-config\") pod \"kube-apiserver-proxy-ip-10-0-128-139.ec2.internal\" (UID: \"7d0a2e4e61221718b5319001dcdbfbbc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.598990 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.598861    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3e3b8db9dd2cfc3e4727a04a3b5eb0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal\" (UID: \"a3e3b8db9dd2cfc3e4727a04a3b5eb0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.598990 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:37.598900    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a3e3b8db9dd2cfc3e4727a04a3b5eb0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal\" (UID: \"a3e3b8db9dd2cfc3e4727a04a3b5eb0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:37.610905 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:37.610869    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-139.ec2.internal\" not found"
Aug 04 18:27:39.157310 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.157278    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-139.ec2.internal\" not found"
Aug 04 18:27:39.243616 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.243567    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:39.249081 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.248891    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:39.258361 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.258345    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-139.ec2.internal\" not found"
Aug 04 18:27:39.349091 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.349008    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:27:39.442432 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.442403    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:39.453109 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.453089    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 18:27:39.454479 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.454464    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal"
Aug 04 18:27:39.467855 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.467832    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 18:27:39.477981 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.477962    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 04 18:27:39.478120 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.478103    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 18:27:39.478156 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.478140    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 18:27:39.515234 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.515212    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:27:39.521426 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.521412    2578 apiserver.go:52] "Watching apiserver"
Aug 04 18:27:39.529987 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.529968    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 04 18:27:39.530339 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.530320    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-9g8kd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal","openshift-multus/multus-486lc","openshift-multus/multus-additional-cni-plugins-wskjj","openshift-multus/network-metrics-daemon-fcf2z","kube-system/konnectivity-agent-hwjp9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd","openshift-cluster-node-tuning-operator/tuned-xsv5g","openshift-network-diagnostics/network-check-target-ds85h","openshift-network-operator/iptables-alerter-ff5v6","openshift-ovn-kubernetes/ovnkube-node-mfgv5","kube-system/global-pull-secret-syncer-h2rx8","kube-system/kube-apiserver-proxy-ip-10-0-128-139.ec2.internal","openshift-dns/node-resolver-8jqq2"]
Aug 04 18:27:39.532874 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.532855    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.534922 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.534905    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.535399 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.535383    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 04 18:27:39.535508 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.535491    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 04 18:27:39.535555 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.535493    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 04 18:27:39.535555 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.535495    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-662c4\""
Aug 04 18:27:39.537036 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.537019    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.537399 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.537379    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 04 18:27:39.537495 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.537384    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nrw74\""
Aug 04 18:27:39.537495 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.537421    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 04 18:27:39.539286 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.539271    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.539535 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.539509    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 04 18:27:39.539535 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.539516    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 04 18:27:39.539692 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.539632    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7scg9\""
Aug 04 18:27:39.539692 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.539655    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 04 18:27:39.539847 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.539832    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 04 18:27:39.541393 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.541379    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 04 18:27:39.541473 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.541393    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:39.541473 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.541454    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:27:39.541729 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.541713    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 04 18:27:39.541804 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.541715    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-46922\""
Aug 04 18:27:39.542095 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.542081    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 04 18:27:39.543502 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.543485    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:27:39.545571 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.545557    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8jqq2"
Aug 04 18:27:39.545842 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.545826    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 04 18:27:39.546008 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.545997    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 04 18:27:39.546075 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.546062    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hjl6l\""
Aug 04 18:27:39.547279 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547264    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-lib-modules\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.547323 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547293    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-var-lib-kubelet\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.547323 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547311    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-run-k8s-cni-cncf-io\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.547401 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-var-lib-cni-bin\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.547401 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547342    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-etc-kubernetes\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.547401 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547385    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-cnibin\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.547501 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547406    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05646d91-5551-4043-8542-b01200ff9771-konnectivity-ca\") pod \"konnectivity-agent-hwjp9\" (UID: \"05646d91-5551-4043-8542-b01200ff9771\") " pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:27:39.547501 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.547501 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-socket-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.547501 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-modprobe-d\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.547501 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547466    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-sysconfig\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.547501 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-systemd\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.547501 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-host\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547505    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-os-release\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvj5n\" (UniqueName: \"kubernetes.io/projected/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-kube-api-access-hvj5n\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547593    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-run\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9rwb\" (UniqueName: \"kubernetes.io/projected/d901f475-84c4-45b1-84cc-402818d94da9-kube-api-access-w9rwb\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547642    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-socket-dir-parent\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547664    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-run-netns\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547689    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-var-lib-cni-multus\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547740    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-system-cni-dir\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547762    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-cni-dir\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547806    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-cnibin\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547833    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-os-release\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.547836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547841    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9g8kd"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8409bf81-5726-4fb2-a0e5-bda061cd3f54-cni-binary-copy\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547877    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-cni-binary-copy\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547919    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547965    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-etc-selinux\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.547997    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-sysctl-d\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548019    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d901f475-84c4-45b1-84cc-402818d94da9-etc-tuned\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548042    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-var-lib-kubelet\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548085    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9vld\" (UniqueName: \"kubernetes.io/projected/8409bf81-5726-4fb2-a0e5-bda061cd3f54-kube-api-access-k9vld\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xw5s\" (UniqueName: \"kubernetes.io/projected/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-kube-api-access-9xw5s\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548164    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-sysctl-conf\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548185    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d901f475-84c4-45b1-84cc-402818d94da9-tmp\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-hostroot\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548227    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-system-cni-dir\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548251    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfs6m\" (UniqueName: \"kubernetes.io/projected/2ef3a825-5bff-4de4-8ce8-4b22a634770c-kube-api-access-dfs6m\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.548392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548324    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-daemon-config\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.548892 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548347    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-sys-fs\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.548892 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548375    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05646d91-5551-4043-8542-b01200ff9771-agent-certs\") pod \"konnectivity-agent-hwjp9\" (UID: \"05646d91-5551-4043-8542-b01200ff9771\") " pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:27:39.548892 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548392    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-registration-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.548892 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548413    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-device-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.548892 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-sys\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.548892 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-conf-dir\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.548892 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-run-multus-certs\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.548892 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548469    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:39.548892 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.548501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-kubernetes\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.549235 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.549219    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 04 18:27:39.549353 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.549337    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 04 18:27:39.549428 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.549344    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-825pn\""
Aug 04 18:27:39.550086 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.550071    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ff5v6"
Aug 04 18:27:39.550267 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.550214    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 04 18:27:39.550343 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.550326    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 04 18:27:39.550431 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.550418    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sgxzk\""
Aug 04 18:27:39.550488 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.550418    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 04 18:27:39.552490 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.552474    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.554843 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.554825    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:39.554922 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.554882    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:27:39.562285 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.562261    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 04 18:27:39.564034 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.563972    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:39.564146 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.564053    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 04 18:27:39.564209 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.564064    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:27:39.564265 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.564231    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 04 18:27:39.564968 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.564123    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-52qd8\""
Aug 04 18:27:39.564968 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.564744    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pwq6c\""
Aug 04 18:27:39.564968 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.564773    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 04 18:27:39.564968 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.564804    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 04 18:27:39.564968 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.564823    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 04 18:27:39.564968 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.564886    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 04 18:27:39.565337 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.565103    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 04 18:27:39.565337 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.565140    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 04 18:27:39.567413 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.567396    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 18:27:39.581612 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.581594    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4v6lr"
Aug 04 18:27:39.592347 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.592325    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4v6lr"
Aug 04 18:27:39.643542 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.643520    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 04 18:27:39.649098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649076    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-sys\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.649222 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-run-multus-certs\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.649222 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649148    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-systemd-units\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.649222 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649191    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2ecaa08-30d0-47a3-b32f-50486a0462e2-dbus\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:39.649222 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da06502a-2382-4c5c-aa50-06912a1cadaf-hosts-file\") pod \"node-resolver-8jqq2\" (UID: \"da06502a-2382-4c5c-aa50-06912a1cadaf\") " pod="openshift-dns/node-resolver-8jqq2"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649226    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-sys\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-kubernetes\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649272    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-run-multus-certs\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-var-lib-cni-bin\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649297    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-etc-kubernetes\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-cnibin\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649323    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-kubernetes\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649355    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-etc-kubernetes\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649362    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-var-lib-cni-bin\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649379    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:39.649382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649383    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-cnibin\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649399    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pksfz\" (UniqueName: \"kubernetes.io/projected/da06502a-2382-4c5c-aa50-06912a1cadaf-kube-api-access-pksfz\") pod \"node-resolver-8jqq2\" (UID: \"da06502a-2382-4c5c-aa50-06912a1cadaf\") " pod="openshift-dns/node-resolver-8jqq2"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.649388    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649421    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05646d91-5551-4043-8542-b01200ff9771-konnectivity-ca\") pod \"konnectivity-agent-hwjp9\" (UID: \"05646d91-5551-4043-8542-b01200ff9771\") " pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649443    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-modprobe-d\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.649483    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs podName:6ebbf61e-7a2b-472f-9232-8d9c81f96ded nodeName:}" failed. No retries permitted until 2026-08-04 18:27:40.149449114 +0000 UTC m=+2.016364216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs") pod "network-metrics-daemon-fcf2z" (UID: "6ebbf61e-7a2b-472f-9232-8d9c81f96ded") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-host\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5391dbb5-02be-4374-b86d-ceda899618d3-iptables-alerter-script\") pod \"iptables-alerter-ff5v6\" (UID: \"5391dbb5-02be-4374-b86d-ceda899618d3\") " pod="openshift-network-operator/iptables-alerter-ff5v6"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649524    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-modprobe-d\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649534    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-run\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-socket-dir-parent\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-host\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649599    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649605    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-run\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649628    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-etc-openvswitch\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649634    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-socket-dir-parent\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcpzh\" (UniqueName: \"kubernetes.io/projected/274adda4-c5aa-448e-baf7-d4b48140929c-kube-api-access-gcpzh\") pod \"node-ca-9g8kd\" (UID: \"274adda4-c5aa-448e-baf7-d4b48140929c\") " pod="openshift-image-registry/node-ca-9g8kd"
Aug 04 18:27:39.649932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-system-cni-dir\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649715    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-cnibin\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649737    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8409bf81-5726-4fb2-a0e5-bda061cd3f54-cni-binary-copy\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649751    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-system-cni-dir\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-run-openvswitch\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649805    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-cnibin\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649814    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-etc-selinux\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-sysctl-d\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d901f475-84c4-45b1-84cc-402818d94da9-etc-tuned\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-run-ovn\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649886    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d901f475-84c4-45b1-84cc-402818d94da9-tmp\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649927    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649964    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.649964    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/853358c5-2d38-4e38-8d7b-d8ea471a92df-ovnkube-config\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650016    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-sysctl-d\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfs6m\" (UniqueName: \"kubernetes.io/projected/2ef3a825-5bff-4de4-8ce8-4b22a634770c-kube-api-access-dfs6m\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.650759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650059    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-daemon-config\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650084    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/274adda4-c5aa-448e-baf7-d4b48140929c-serviceca\") pod \"node-ca-9g8kd\" (UID: \"274adda4-c5aa-448e-baf7-d4b48140929c\") " pod="openshift-image-registry/node-ca-9g8kd"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650093    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05646d91-5551-4043-8542-b01200ff9771-konnectivity-ca\") pod \"konnectivity-agent-hwjp9\" (UID: \"05646d91-5551-4043-8542-b01200ff9771\") " pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-sys-fs\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-etc-selinux\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05646d91-5551-4043-8542-b01200ff9771-agent-certs\") pod \"konnectivity-agent-hwjp9\" (UID: \"05646d91-5551-4043-8542-b01200ff9771\") " pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650153    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650172    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-registration-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-device-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-conf-dir\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650245    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-registration-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5391dbb5-02be-4374-b86d-ceda899618d3-host-slash\") pod \"iptables-alerter-ff5v6\" (UID: \"5391dbb5-02be-4374-b86d-ceda899618d3\") " pod="openshift-network-operator/iptables-alerter-ff5v6"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650282    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650295    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-kubelet\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650301    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-sys-fs\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650316    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8409bf81-5726-4fb2-a0e5-bda061cd3f54-cni-binary-copy\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-lib-modules\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.651529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-var-lib-kubelet\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650369    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-run-k8s-cni-cncf-io\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650366    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-device-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-cni-binary-copy\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-conf-dir\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650446    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-var-lib-kubelet\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650475    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-run-k8s-cni-cncf-io\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650449    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-var-lib-openvswitch\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-log-socket\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650550    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-cni-netd\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650571    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/853358c5-2d38-4e38-8d7b-d8ea471a92df-ovnkube-script-lib\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650611    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-lib-modules\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-socket-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650651    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-daemon-config\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-sysconfig\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.652352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650686    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-systemd\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650694    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-os-release\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da06502a-2382-4c5c-aa50-06912a1cadaf-tmp-dir\") pod \"node-resolver-8jqq2\" (UID: \"da06502a-2382-4c5c-aa50-06912a1cadaf\") " pod="openshift-dns/node-resolver-8jqq2"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650758    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-sysconfig\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650764    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2ef3a825-5bff-4de4-8ce8-4b22a634770c-socket-dir\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650767    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-run-netns\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650809    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-run-systemd\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650817    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-os-release\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650856    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-systemd\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650856    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvj5n\" (UniqueName: \"kubernetes.io/projected/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-kube-api-access-hvj5n\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9rwb\" (UniqueName: \"kubernetes.io/projected/d901f475-84c4-45b1-84cc-402818d94da9-kube-api-access-w9rwb\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650923    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-cni-binary-copy\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650933    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-run-netns\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650974    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-var-lib-cni-multus\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.650997    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-cni-bin\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651019    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/853358c5-2d38-4e38-8d7b-d8ea471a92df-env-overrides\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.653192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651042    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmklb\" (UniqueName: \"kubernetes.io/projected/853358c5-2d38-4e38-8d7b-d8ea471a92df-kube-api-access-xmklb\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651067    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/274adda4-c5aa-448e-baf7-d4b48140929c-host\") pod \"node-ca-9g8kd\" (UID: \"274adda4-c5aa-448e-baf7-d4b48140929c\") " pod="openshift-image-registry/node-ca-9g8kd"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651094    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xw5s\" (UniqueName: \"kubernetes.io/projected/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-kube-api-access-9xw5s\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-cni-dir\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-os-release\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651137    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-run-netns\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651180    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-var-lib-cni-multus\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651187    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-slash\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651210    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-node-log\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651213    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-os-release\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2ecaa08-30d0-47a3-b32f-50486a0462e2-kubelet-config\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651259    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w2j2\" (UniqueName: \"kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2\") pod \"network-check-target-ds85h\" (UID: \"07c78662-4da8-47a9-b9ec-f03b7945ff62\") " pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-var-lib-kubelet\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651314    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-host-var-lib-kubelet\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651317    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-multus-cni-dir\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9vld\" (UniqueName: \"kubernetes.io/projected/8409bf81-5726-4fb2-a0e5-bda061cd3f54-kube-api-access-k9vld\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-sysctl-conf\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.653955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651390    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-hostroot\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.654463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-system-cni-dir\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.654463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651461    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdtnj\" (UniqueName: \"kubernetes.io/projected/5391dbb5-02be-4374-b86d-ceda899618d3-kube-api-access-vdtnj\") pod \"iptables-alerter-ff5v6\" (UID: \"5391dbb5-02be-4374-b86d-ceda899618d3\") " pod="openshift-network-operator/iptables-alerter-ff5v6"
Aug 04 18:27:39.654463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651490    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-run-ovn-kubernetes\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.654463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651515    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.654463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/853358c5-2d38-4e38-8d7b-d8ea471a92df-ovn-node-metrics-cert\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.654463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651552    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-system-cni-dir\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.654463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651518    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d901f475-84c4-45b1-84cc-402818d94da9-etc-sysctl-conf\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.654463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.651651    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8409bf81-5726-4fb2-a0e5-bda061cd3f54-hostroot\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.654463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.653820    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d901f475-84c4-45b1-84cc-402818d94da9-etc-tuned\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.654463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.653969    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d901f475-84c4-45b1-84cc-402818d94da9-tmp\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.654463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.654100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05646d91-5551-4043-8542-b01200ff9771-agent-certs\") pod \"konnectivity-agent-hwjp9\" (UID: \"05646d91-5551-4043-8542-b01200ff9771\") " pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:27:39.661761 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.661444    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfs6m\" (UniqueName: \"kubernetes.io/projected/2ef3a825-5bff-4de4-8ce8-4b22a634770c-kube-api-access-dfs6m\") pod \"aws-ebs-csi-driver-node-s2bxd\" (UID: \"2ef3a825-5bff-4de4-8ce8-4b22a634770c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.663456 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.663429    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvj5n\" (UniqueName: \"kubernetes.io/projected/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-kube-api-access-hvj5n\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:39.664244 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.664222    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9rwb\" (UniqueName: \"kubernetes.io/projected/d901f475-84c4-45b1-84cc-402818d94da9-kube-api-access-w9rwb\") pod \"tuned-xsv5g\" (UID: \"d901f475-84c4-45b1-84cc-402818d94da9\") " pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.664380 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.664352    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9vld\" (UniqueName: \"kubernetes.io/projected/8409bf81-5726-4fb2-a0e5-bda061cd3f54-kube-api-access-k9vld\") pod \"multus-486lc\" (UID: \"8409bf81-5726-4fb2-a0e5-bda061cd3f54\") " pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.664440 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.664409    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xw5s\" (UniqueName: \"kubernetes.io/projected/3f1e88cb-747e-47e7-a1fe-bfd764c13f39-kube-api-access-9xw5s\") pod \"multus-additional-cni-plugins-wskjj\" (UID: \"3f1e88cb-747e-47e7-a1fe-bfd764c13f39\") " pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.710850 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.710797    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 18:22:38 +0000 UTC" deadline="2028-04-04 06:56:34.323905716 +0000 UTC"
Aug 04 18:27:39.710850 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.710840    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14604h28m54.613068766s"
Aug 04 18:27:39.752505 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752475    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da06502a-2382-4c5c-aa50-06912a1cadaf-hosts-file\") pod \"node-resolver-8jqq2\" (UID: \"da06502a-2382-4c5c-aa50-06912a1cadaf\") " pod="openshift-dns/node-resolver-8jqq2"
Aug 04 18:27:39.752682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752534    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:39.752682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pksfz\" (UniqueName: \"kubernetes.io/projected/da06502a-2382-4c5c-aa50-06912a1cadaf-kube-api-access-pksfz\") pod \"node-resolver-8jqq2\" (UID: \"da06502a-2382-4c5c-aa50-06912a1cadaf\") " pod="openshift-dns/node-resolver-8jqq2"
Aug 04 18:27:39.752682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752612    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5391dbb5-02be-4374-b86d-ceda899618d3-iptables-alerter-script\") pod \"iptables-alerter-ff5v6\" (UID: \"5391dbb5-02be-4374-b86d-ceda899618d3\") " pod="openshift-network-operator/iptables-alerter-ff5v6"
Aug 04 18:27:39.752682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752625    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/da06502a-2382-4c5c-aa50-06912a1cadaf-hosts-file\") pod \"node-resolver-8jqq2\" (UID: \"da06502a-2382-4c5c-aa50-06912a1cadaf\") " pod="openshift-dns/node-resolver-8jqq2"
Aug 04 18:27:39.752682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752639    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-etc-openvswitch\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.752860 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752685    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-etc-openvswitch\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.752860 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752686    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcpzh\" (UniqueName: \"kubernetes.io/projected/274adda4-c5aa-448e-baf7-d4b48140929c-kube-api-access-gcpzh\") pod \"node-ca-9g8kd\" (UID: \"274adda4-c5aa-448e-baf7-d4b48140929c\") " pod="openshift-image-registry/node-ca-9g8kd"
Aug 04 18:27:39.752860 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.752714    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:39.752860 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.752790    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret podName:d2ecaa08-30d0-47a3-b32f-50486a0462e2 nodeName:}" failed. No retries permitted until 2026-08-04 18:27:40.252769817 +0000 UTC m=+2.119684910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret") pod "global-pull-secret-syncer-h2rx8" (UID: "d2ecaa08-30d0-47a3-b32f-50486a0462e2") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:39.752860 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752823    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-run-openvswitch\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.752860 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-run-ovn\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.752860 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752857    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-run-openvswitch\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752882    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/853358c5-2d38-4e38-8d7b-d8ea471a92df-ovnkube-config\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752893    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-run-ovn\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752909    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/274adda4-c5aa-448e-baf7-d4b48140929c-serviceca\") pod \"node-ca-9g8kd\" (UID: \"274adda4-c5aa-448e-baf7-d4b48140929c\") " pod="openshift-image-registry/node-ca-9g8kd"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752939    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5391dbb5-02be-4374-b86d-ceda899618d3-host-slash\") pod \"iptables-alerter-ff5v6\" (UID: \"5391dbb5-02be-4374-b86d-ceda899618d3\") " pod="openshift-network-operator/iptables-alerter-ff5v6"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752962    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-kubelet\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752988    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-var-lib-openvswitch\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.752997    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5391dbb5-02be-4374-b86d-ceda899618d3-host-slash\") pod \"iptables-alerter-ff5v6\" (UID: \"5391dbb5-02be-4374-b86d-ceda899618d3\") " pod="openshift-network-operator/iptables-alerter-ff5v6"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753012    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-log-socket\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-kubelet\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-cni-netd\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/853358c5-2d38-4e38-8d7b-d8ea471a92df-ovnkube-script-lib\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753065    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-var-lib-openvswitch\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da06502a-2382-4c5c-aa50-06912a1cadaf-tmp-dir\") pod \"node-resolver-8jqq2\" (UID: \"da06502a-2382-4c5c-aa50-06912a1cadaf\") " pod="openshift-dns/node-resolver-8jqq2"
Aug 04 18:27:39.753094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753088    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-log-socket\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753105    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-run-netns\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753128    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-run-systemd\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753127    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-cni-netd\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753159    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-cni-bin\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/853358c5-2d38-4e38-8d7b-d8ea471a92df-env-overrides\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753223    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmklb\" (UniqueName: \"kubernetes.io/projected/853358c5-2d38-4e38-8d7b-d8ea471a92df-kube-api-access-xmklb\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753234    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-run-netns\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/274adda4-c5aa-448e-baf7-d4b48140929c-host\") pod \"node-ca-9g8kd\" (UID: \"274adda4-c5aa-448e-baf7-d4b48140929c\") " pod="openshift-image-registry/node-ca-9g8kd"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753272    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-slash\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753284    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-cni-bin\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753299    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-node-log\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753330    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2ecaa08-30d0-47a3-b32f-50486a0462e2-kubelet-config\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753347    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-run-systemd\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w2j2\" (UniqueName: \"kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2\") pod \"network-check-target-ds85h\" (UID: \"07c78662-4da8-47a9-b9ec-f03b7945ff62\") " pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753383    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da06502a-2382-4c5c-aa50-06912a1cadaf-tmp-dir\") pod \"node-resolver-8jqq2\" (UID: \"da06502a-2382-4c5c-aa50-06912a1cadaf\") " pod="openshift-dns/node-resolver-8jqq2"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753401    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/274adda4-c5aa-448e-baf7-d4b48140929c-host\") pod \"node-ca-9g8kd\" (UID: \"274adda4-c5aa-448e-baf7-d4b48140929c\") " pod="openshift-image-registry/node-ca-9g8kd"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753413    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-node-log\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.753723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753385    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdtnj\" (UniqueName: \"kubernetes.io/projected/5391dbb5-02be-4374-b86d-ceda899618d3-kube-api-access-vdtnj\") pod \"iptables-alerter-ff5v6\" (UID: \"5391dbb5-02be-4374-b86d-ceda899618d3\") " pod="openshift-network-operator/iptables-alerter-ff5v6"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753441    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2ecaa08-30d0-47a3-b32f-50486a0462e2-kubelet-config\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753462    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-slash\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753461    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/274adda4-c5aa-448e-baf7-d4b48140929c-serviceca\") pod \"node-ca-9g8kd\" (UID: \"274adda4-c5aa-448e-baf7-d4b48140929c\") " pod="openshift-image-registry/node-ca-9g8kd"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-run-ovn-kubernetes\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753480    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/853358c5-2d38-4e38-8d7b-d8ea471a92df-ovnkube-config\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753520    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-run-ovn-kubernetes\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753553    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/853358c5-2d38-4e38-8d7b-d8ea471a92df-ovn-node-metrics-cert\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753604    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/853358c5-2d38-4e38-8d7b-d8ea471a92df-ovnkube-script-lib\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753627    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753705    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-systemd-units\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753735    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2ecaa08-30d0-47a3-b32f-50486a0462e2-dbus\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753795    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/853358c5-2d38-4e38-8d7b-d8ea471a92df-systemd-units\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5391dbb5-02be-4374-b86d-ceda899618d3-iptables-alerter-script\") pod \"iptables-alerter-ff5v6\" (UID: \"5391dbb5-02be-4374-b86d-ceda899618d3\") " pod="openshift-network-operator/iptables-alerter-ff5v6"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753814    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2ecaa08-30d0-47a3-b32f-50486a0462e2-dbus\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:39.754266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.753838    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/853358c5-2d38-4e38-8d7b-d8ea471a92df-env-overrides\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.755613 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.755599    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/853358c5-2d38-4e38-8d7b-d8ea471a92df-ovn-node-metrics-cert\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.760539 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.760518    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:27:39.760652 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.760545    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:27:39.760652 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.760557    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9w2j2 for pod openshift-network-diagnostics/network-check-target-ds85h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:39.760773 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:39.760654    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2 podName:07c78662-4da8-47a9-b9ec-f03b7945ff62 nodeName:}" failed. No retries permitted until 2026-08-04 18:27:40.260634658 +0000 UTC m=+2.127549744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9w2j2" (UniqueName: "kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2") pod "network-check-target-ds85h" (UID: "07c78662-4da8-47a9-b9ec-f03b7945ff62") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:39.762377 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.762355    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcpzh\" (UniqueName: \"kubernetes.io/projected/274adda4-c5aa-448e-baf7-d4b48140929c-kube-api-access-gcpzh\") pod \"node-ca-9g8kd\" (UID: \"274adda4-c5aa-448e-baf7-d4b48140929c\") " pod="openshift-image-registry/node-ca-9g8kd"
Aug 04 18:27:39.762566 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.762548    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pksfz\" (UniqueName: \"kubernetes.io/projected/da06502a-2382-4c5c-aa50-06912a1cadaf-kube-api-access-pksfz\") pod \"node-resolver-8jqq2\" (UID: \"da06502a-2382-4c5c-aa50-06912a1cadaf\") " pod="openshift-dns/node-resolver-8jqq2"
Aug 04 18:27:39.762625 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.762550    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdtnj\" (UniqueName: \"kubernetes.io/projected/5391dbb5-02be-4374-b86d-ceda899618d3-kube-api-access-vdtnj\") pod \"iptables-alerter-ff5v6\" (UID: \"5391dbb5-02be-4374-b86d-ceda899618d3\") " pod="openshift-network-operator/iptables-alerter-ff5v6"
Aug 04 18:27:39.762997 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.762983    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmklb\" (UniqueName: \"kubernetes.io/projected/853358c5-2d38-4e38-8d7b-d8ea471a92df-kube-api-access-xmklb\") pod \"ovnkube-node-mfgv5\" (UID: \"853358c5-2d38-4e38-8d7b-d8ea471a92df\") " pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:39.871126 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.871055    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd"
Aug 04 18:27:39.877233 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.877213    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xsv5g"
Aug 04 18:27:39.892696 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.892672    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:27:39.894519 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.894495    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-486lc"
Aug 04 18:27:39.908571 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.908535    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wskjj"
Aug 04 18:27:39.923666 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.923640    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:27:39.937448 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.937422    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8jqq2"
Aug 04 18:27:39.953153 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.953124    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9g8kd"
Aug 04 18:27:39.980749 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.980719    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ff5v6"
Aug 04 18:27:39.986299 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:39.986282    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:27:40.006556 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:40.006509    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod274adda4_c5aa_448e_baf7_d4b48140929c.slice/crio-fc1e172bb2a98d9b48b1c151e16e6fc09487cc37240f017254dbf04a3720dd8e WatchSource:0}: Error finding container fc1e172bb2a98d9b48b1c151e16e6fc09487cc37240f017254dbf04a3720dd8e: Status 404 returned error can't find the container with id fc1e172bb2a98d9b48b1c151e16e6fc09487cc37240f017254dbf04a3720dd8e
Aug 04 18:27:40.007023 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:40.007002    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853358c5_2d38_4e38_8d7b_d8ea471a92df.slice/crio-06bc4c4eafa9e0e50046354487e310868fb7e99a1045d7c4e6b7b669dd719db0 WatchSource:0}: Error finding container 06bc4c4eafa9e0e50046354487e310868fb7e99a1045d7c4e6b7b669dd719db0: Status 404 returned error can't find the container with id 06bc4c4eafa9e0e50046354487e310868fb7e99a1045d7c4e6b7b669dd719db0
Aug 04 18:27:40.007843 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:40.007825    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e3b8db9dd2cfc3e4727a04a3b5eb0f.slice/crio-5a45d23d074f2c46674b6bcc930fd0daab3a3a25a6c2d063300f6baa1d194d7d WatchSource:0}: Error finding container 5a45d23d074f2c46674b6bcc930fd0daab3a3a25a6c2d063300f6baa1d194d7d: Status 404 returned error can't find the container with id 5a45d23d074f2c46674b6bcc930fd0daab3a3a25a6c2d063300f6baa1d194d7d
Aug 04 18:27:40.008491 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:40.008464    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0a2e4e61221718b5319001dcdbfbbc.slice/crio-7c69fff6cdf8a60c408a10631d83cc45a35dab1307912cb0ec800adb7a981253 WatchSource:0}: Error finding container 7c69fff6cdf8a60c408a10631d83cc45a35dab1307912cb0ec800adb7a981253: Status 404 returned error can't find the container with id 7c69fff6cdf8a60c408a10631d83cc45a35dab1307912cb0ec800adb7a981253
Aug 04 18:27:40.009485 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:40.009467    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5391dbb5_02be_4374_b86d_ceda899618d3.slice/crio-e5470f5e39a6ea1f67d1835c94f75ad93e00d886e3a446aa42319809bf48eb27 WatchSource:0}: Error finding container e5470f5e39a6ea1f67d1835c94f75ad93e00d886e3a446aa42319809bf48eb27: Status 404 returned error can't find the container with id e5470f5e39a6ea1f67d1835c94f75ad93e00d886e3a446aa42319809bf48eb27
Aug 04 18:27:40.012716 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.012646    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 18:27:40.156032 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.155934    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:40.156176 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:40.156099    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:40.156176 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:40.156164    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs podName:6ebbf61e-7a2b-472f-9232-8d9c81f96ded nodeName:}" failed. No retries permitted until 2026-08-04 18:27:41.156149759 +0000 UTC m=+3.023064850 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs") pod "network-metrics-daemon-fcf2z" (UID: "6ebbf61e-7a2b-472f-9232-8d9c81f96ded") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:40.209318 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:40.209286    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05646d91_5551_4043_8542_b01200ff9771.slice/crio-b14ad46b21c6bfb906038ca88a2cb8cfa1c8fc96f047f6a316ce7e9a657fe370 WatchSource:0}: Error finding container b14ad46b21c6bfb906038ca88a2cb8cfa1c8fc96f047f6a316ce7e9a657fe370: Status 404 returned error can't find the container with id b14ad46b21c6bfb906038ca88a2cb8cfa1c8fc96f047f6a316ce7e9a657fe370
Aug 04 18:27:40.257207 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.257174    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:40.257382 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:40.257296    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:40.257382 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:40.257348    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret podName:d2ecaa08-30d0-47a3-b32f-50486a0462e2 nodeName:}" failed. No retries permitted until 2026-08-04 18:27:41.257333433 +0000 UTC m=+3.124248512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret") pod "global-pull-secret-syncer-h2rx8" (UID: "d2ecaa08-30d0-47a3-b32f-50486a0462e2") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:40.293853 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:40.293820    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda06502a_2382_4c5c_aa50_06912a1cadaf.slice/crio-ebb25147ebde9e39f69da00e47268cf74b02959bd6458989af4e8e7619970c6e WatchSource:0}: Error finding container ebb25147ebde9e39f69da00e47268cf74b02959bd6458989af4e8e7619970c6e: Status 404 returned error can't find the container with id ebb25147ebde9e39f69da00e47268cf74b02959bd6458989af4e8e7619970c6e
Aug 04 18:27:40.313812 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:40.313782    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd901f475_84c4_45b1_84cc_402818d94da9.slice/crio-5402fe2df75a4ff75e4aa2e5e812d7324526afdeb59d29d50ac31f114d59667e WatchSource:0}: Error finding container 5402fe2df75a4ff75e4aa2e5e812d7324526afdeb59d29d50ac31f114d59667e: Status 404 returned error can't find the container with id 5402fe2df75a4ff75e4aa2e5e812d7324526afdeb59d29d50ac31f114d59667e
Aug 04 18:27:40.352632 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:40.352605    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f1e88cb_747e_47e7_a1fe_bfd764c13f39.slice/crio-8d812d06d470fc10c44d54ce709fb2f36407c077eecc05f099b639b89c34d255 WatchSource:0}: Error finding container 8d812d06d470fc10c44d54ce709fb2f36407c077eecc05f099b639b89c34d255: Status 404 returned error can't find the container with id 8d812d06d470fc10c44d54ce709fb2f36407c077eecc05f099b639b89c34d255
Aug 04 18:27:40.358063 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.358038    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w2j2\" (UniqueName: \"kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2\") pod \"network-check-target-ds85h\" (UID: \"07c78662-4da8-47a9-b9ec-f03b7945ff62\") " pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:40.358198 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:40.358180    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:27:40.358237 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:40.358204    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:27:40.358237 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:40.358214    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9w2j2 for pod openshift-network-diagnostics/network-check-target-ds85h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:40.358297 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:40.358257    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2 podName:07c78662-4da8-47a9-b9ec-f03b7945ff62 nodeName:}" failed. No retries permitted until 2026-08-04 18:27:41.358243287 +0000 UTC m=+3.225158366 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w2j2" (UniqueName: "kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2") pod "network-check-target-ds85h" (UID: "07c78662-4da8-47a9-b9ec-f03b7945ff62") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:40.373742 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:40.373709    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8409bf81_5726_4fb2_a0e5_bda061cd3f54.slice/crio-b1178bbae6e3c45bf7df1746e15eb55c249fcb22d79dc1b87696471e1c4f1fda WatchSource:0}: Error finding container b1178bbae6e3c45bf7df1746e15eb55c249fcb22d79dc1b87696471e1c4f1fda: Status 404 returned error can't find the container with id b1178bbae6e3c45bf7df1746e15eb55c249fcb22d79dc1b87696471e1c4f1fda
Aug 04 18:27:40.413189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.413106    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:27:40.596165 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.596057    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 18:22:39 +0000 UTC" deadline="2028-02-26 10:36:55.188533645 +0000 UTC"
Aug 04 18:27:40.596165 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.596169    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13696h9m14.592373654s"
Aug 04 18:27:40.607274 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:27:40.607244    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ef3a825_5bff_4de4_8ce8_4b22a634770c.slice/crio-bfbd142a39e07d2b7e4483a8bb4b0b88323a1f68fd43ae731b92cff4c3b3fc09 WatchSource:0}: Error finding container bfbd142a39e07d2b7e4483a8bb4b0b88323a1f68fd43ae731b92cff4c3b3fc09: Status 404 returned error can't find the container with id bfbd142a39e07d2b7e4483a8bb4b0b88323a1f68fd43ae731b92cff4c3b3fc09
Aug 04 18:27:40.797063 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.796978    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:40.797232 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.796986    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:40.797232 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:40.797134    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:27:40.797232 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:40.797177    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:27:40.805647 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.805569    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5" event={"ID":"853358c5-2d38-4e38-8d7b-d8ea471a92df","Type":"ContainerStarted","Data":"06bc4c4eafa9e0e50046354487e310868fb7e99a1045d7c4e6b7b669dd719db0"}
Aug 04 18:27:40.812554 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.812516    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal" event={"ID":"a3e3b8db9dd2cfc3e4727a04a3b5eb0f","Type":"ContainerStarted","Data":"5a45d23d074f2c46674b6bcc930fd0daab3a3a25a6c2d063300f6baa1d194d7d"}
Aug 04 18:27:40.825322 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.825283    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ff5v6" event={"ID":"5391dbb5-02be-4374-b86d-ceda899618d3","Type":"ContainerStarted","Data":"e5470f5e39a6ea1f67d1835c94f75ad93e00d886e3a446aa42319809bf48eb27"}
Aug 04 18:27:40.831366 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.831326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9g8kd" event={"ID":"274adda4-c5aa-448e-baf7-d4b48140929c","Type":"ContainerStarted","Data":"fc1e172bb2a98d9b48b1c151e16e6fc09487cc37240f017254dbf04a3720dd8e"}
Aug 04 18:27:40.838462 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.838422    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-486lc" event={"ID":"8409bf81-5726-4fb2-a0e5-bda061cd3f54","Type":"ContainerStarted","Data":"b1178bbae6e3c45bf7df1746e15eb55c249fcb22d79dc1b87696471e1c4f1fda"}
Aug 04 18:27:40.843319 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.843285    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xsv5g" event={"ID":"d901f475-84c4-45b1-84cc-402818d94da9","Type":"ContainerStarted","Data":"5402fe2df75a4ff75e4aa2e5e812d7324526afdeb59d29d50ac31f114d59667e"}
Aug 04 18:27:40.851003 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.850965    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8jqq2" event={"ID":"da06502a-2382-4c5c-aa50-06912a1cadaf","Type":"ContainerStarted","Data":"ebb25147ebde9e39f69da00e47268cf74b02959bd6458989af4e8e7619970c6e"}
Aug 04 18:27:40.856135 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.856089    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-139.ec2.internal" event={"ID":"7d0a2e4e61221718b5319001dcdbfbbc","Type":"ContainerStarted","Data":"7c69fff6cdf8a60c408a10631d83cc45a35dab1307912cb0ec800adb7a981253"}
Aug 04 18:27:40.858085 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.858046    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd" event={"ID":"2ef3a825-5bff-4de4-8ce8-4b22a634770c","Type":"ContainerStarted","Data":"bfbd142a39e07d2b7e4483a8bb4b0b88323a1f68fd43ae731b92cff4c3b3fc09"}
Aug 04 18:27:40.867999 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.867920    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wskjj" event={"ID":"3f1e88cb-747e-47e7-a1fe-bfd764c13f39","Type":"ContainerStarted","Data":"8d812d06d470fc10c44d54ce709fb2f36407c077eecc05f099b639b89c34d255"}
Aug 04 18:27:40.873801 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:40.873765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hwjp9" event={"ID":"05646d91-5551-4043-8542-b01200ff9771","Type":"ContainerStarted","Data":"b14ad46b21c6bfb906038ca88a2cb8cfa1c8fc96f047f6a316ce7e9a657fe370"}
Aug 04 18:27:41.163829 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:41.163729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:41.164016 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:41.163907    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:41.164096 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:41.164055    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs podName:6ebbf61e-7a2b-472f-9232-8d9c81f96ded nodeName:}" failed. No retries permitted until 2026-08-04 18:27:43.163965491 +0000 UTC m=+5.030880586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs") pod "network-metrics-daemon-fcf2z" (UID: "6ebbf61e-7a2b-472f-9232-8d9c81f96ded") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:41.265466 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:41.264823    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:41.265466 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:41.265039    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:41.265466 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:41.265104    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret podName:d2ecaa08-30d0-47a3-b32f-50486a0462e2 nodeName:}" failed. No retries permitted until 2026-08-04 18:27:43.265085146 +0000 UTC m=+5.132000229 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret") pod "global-pull-secret-syncer-h2rx8" (UID: "d2ecaa08-30d0-47a3-b32f-50486a0462e2") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:41.366953 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:41.366229    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w2j2\" (UniqueName: \"kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2\") pod \"network-check-target-ds85h\" (UID: \"07c78662-4da8-47a9-b9ec-f03b7945ff62\") " pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:41.366953 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:41.366450    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:27:41.366953 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:41.366470    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:27:41.366953 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:41.366495    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9w2j2 for pod openshift-network-diagnostics/network-check-target-ds85h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:41.366953 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:41.366557    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2 podName:07c78662-4da8-47a9-b9ec-f03b7945ff62 nodeName:}" failed. No retries permitted until 2026-08-04 18:27:43.366538385 +0000 UTC m=+5.233453487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w2j2" (UniqueName: "kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2") pod "network-check-target-ds85h" (UID: "07c78662-4da8-47a9-b9ec-f03b7945ff62") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:41.597044 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:41.596939    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 18:22:39 +0000 UTC" deadline="2028-01-21 19:24:35.733324843 +0000 UTC"
Aug 04 18:27:41.597044 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:41.596989    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12840h56m54.136343413s"
Aug 04 18:27:41.797436 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:41.796914    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:41.797436 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:41.797061    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:27:42.638142 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:42.638102    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 18:27:42.799594 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:42.796081    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:42.799594 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:42.796218    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:27:42.799594 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:42.796028    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:42.799594 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:42.796720    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:27:43.187365 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:43.187281    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:43.187534 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:43.187461    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:43.187609 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:43.187535    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs podName:6ebbf61e-7a2b-472f-9232-8d9c81f96ded nodeName:}" failed. No retries permitted until 2026-08-04 18:27:47.187515575 +0000 UTC m=+9.054430668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs") pod "network-metrics-daemon-fcf2z" (UID: "6ebbf61e-7a2b-472f-9232-8d9c81f96ded") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:43.288445 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:43.288402    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:43.288657 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:43.288602    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:43.288719 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:43.288666    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret podName:d2ecaa08-30d0-47a3-b32f-50486a0462e2 nodeName:}" failed. No retries permitted until 2026-08-04 18:27:47.28864814 +0000 UTC m=+9.155563219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret") pod "global-pull-secret-syncer-h2rx8" (UID: "d2ecaa08-30d0-47a3-b32f-50486a0462e2") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:43.389019 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:43.388979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w2j2\" (UniqueName: \"kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2\") pod \"network-check-target-ds85h\" (UID: \"07c78662-4da8-47a9-b9ec-f03b7945ff62\") " pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:43.389202 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:43.389132    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:27:43.389202 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:43.389155    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:27:43.389202 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:43.389167    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9w2j2 for pod openshift-network-diagnostics/network-check-target-ds85h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:43.389355 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:43.389222    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2 podName:07c78662-4da8-47a9-b9ec-f03b7945ff62 nodeName:}" failed. No retries permitted until 2026-08-04 18:27:47.389204508 +0000 UTC m=+9.256119602 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w2j2" (UniqueName: "kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2") pod "network-check-target-ds85h" (UID: "07c78662-4da8-47a9-b9ec-f03b7945ff62") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:43.796712 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:43.796679    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:43.797205 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:43.796810    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:27:44.796586 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:44.796538    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:44.796771 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:44.796538    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:44.796771 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:44.796701    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:27:44.797196 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:44.796797    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:27:45.796531 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:45.796494    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:45.796737 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:45.796642    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:27:46.796502 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:46.796469    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:46.797005 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:46.796627    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:27:46.797108 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:46.797090    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:46.797214 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:46.797192    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:27:47.221663 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:47.221507    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:47.221808 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:47.221706    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:47.221808 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:47.221792    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs podName:6ebbf61e-7a2b-472f-9232-8d9c81f96ded nodeName:}" failed. No retries permitted until 2026-08-04 18:27:55.221771037 +0000 UTC m=+17.088686120 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs") pod "network-metrics-daemon-fcf2z" (UID: "6ebbf61e-7a2b-472f-9232-8d9c81f96ded") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:47.322767 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:47.322726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:47.322956 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:47.322889    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:47.322956 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:47.322949    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret podName:d2ecaa08-30d0-47a3-b32f-50486a0462e2 nodeName:}" failed. No retries permitted until 2026-08-04 18:27:55.322932324 +0000 UTC m=+17.189847420 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret") pod "global-pull-secret-syncer-h2rx8" (UID: "d2ecaa08-30d0-47a3-b32f-50486a0462e2") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:47.423363 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:47.423233    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w2j2\" (UniqueName: \"kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2\") pod \"network-check-target-ds85h\" (UID: \"07c78662-4da8-47a9-b9ec-f03b7945ff62\") " pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:47.423550 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:47.423446    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:27:47.423550 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:47.423473    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:27:47.423550 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:47.423487    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9w2j2 for pod openshift-network-diagnostics/network-check-target-ds85h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:47.423550 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:47.423549    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2 podName:07c78662-4da8-47a9-b9ec-f03b7945ff62 nodeName:}" failed. No retries permitted until 2026-08-04 18:27:55.423528651 +0000 UTC m=+17.290443747 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w2j2" (UniqueName: "kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2") pod "network-check-target-ds85h" (UID: "07c78662-4da8-47a9-b9ec-f03b7945ff62") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:47.796281 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:47.796246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:47.796481 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:47.796358    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:27:48.797477 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:48.797433    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:48.797953 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:48.797557    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:27:48.797953 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:48.797622    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:48.797953 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:48.797775    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:27:49.796309 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:49.796269    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:49.796503 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:49.796415    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:27:50.796563 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:50.796345    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:50.797337 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:50.796684    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:27:50.797337 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:50.796431    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:50.797337 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:50.796812    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:27:51.796106 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:51.796078    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:51.796233 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:51.796179    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:27:52.796291 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:52.796253    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:52.796782 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:52.796252    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:52.796782 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:52.796388    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:27:52.796782 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:52.796449    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:27:53.796525 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:53.796485    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:53.796947 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:53.796620    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:27:54.796380 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:54.796347    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:54.796553 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:54.796347    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:54.796553 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:54.796481    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:27:54.796953 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:54.796557    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:27:55.286721 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:55.286689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:55.286898 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:55.286845    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:55.286986 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:55.286917    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs podName:6ebbf61e-7a2b-472f-9232-8d9c81f96ded nodeName:}" failed. No retries permitted until 2026-08-04 18:28:11.28690256 +0000 UTC m=+33.153817643 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs") pod "network-metrics-daemon-fcf2z" (UID: "6ebbf61e-7a2b-472f-9232-8d9c81f96ded") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:27:55.387965 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:55.387917    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:55.388144 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:55.388113    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:55.388214 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:55.388186    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret podName:d2ecaa08-30d0-47a3-b32f-50486a0462e2 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:11.388165064 +0000 UTC m=+33.255080153 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret") pod "global-pull-secret-syncer-h2rx8" (UID: "d2ecaa08-30d0-47a3-b32f-50486a0462e2") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:27:55.489255 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:55.489219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w2j2\" (UniqueName: \"kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2\") pod \"network-check-target-ds85h\" (UID: \"07c78662-4da8-47a9-b9ec-f03b7945ff62\") " pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:55.489412 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:55.489393    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:27:55.489460 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:55.489417    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:27:55.489460 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:55.489428    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9w2j2 for pod openshift-network-diagnostics/network-check-target-ds85h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:55.489516 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:55.489481    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2 podName:07c78662-4da8-47a9-b9ec-f03b7945ff62 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:11.489467442 +0000 UTC m=+33.356382521 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w2j2" (UniqueName: "kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2") pod "network-check-target-ds85h" (UID: "07c78662-4da8-47a9-b9ec-f03b7945ff62") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:27:55.796546 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:55.796514    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:55.796993 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:55.796636    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:27:56.796424 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:56.796381    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:56.796630 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:56.796436    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:56.796630 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:56.796518    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:27:56.796992 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:56.796655    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:27:57.796438 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:57.796400    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:57.796645 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:57.796529    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:27:58.797186 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:58.797146    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:27:58.797625 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:58.797268    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:27:58.797625 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:58.797323    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:27:58.797625 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:58.797449    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:27:59.795965 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:27:59.795925    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:27:59.796124 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:27:59.796041    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:28:00.796759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.796546    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:28:00.797623 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.796587    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:00.797623 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:00.796871    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:28:00.797623 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:00.796903    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:28:00.923966 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.923930    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-486lc" event={"ID":"8409bf81-5726-4fb2-a0e5-bda061cd3f54","Type":"ContainerStarted","Data":"a8d926145d6baf05b591f80307da7fbf1921cdfdba8cbe44e81dc0371d6ad9d5"}
Aug 04 18:28:00.925599 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.925551    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xsv5g" event={"ID":"d901f475-84c4-45b1-84cc-402818d94da9","Type":"ContainerStarted","Data":"9781d7945c7ad08b3f2c92b17d9d31fe54a01b3b3f16f2e39abb6e5206a8c964"}
Aug 04 18:28:00.926966 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.926941    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8jqq2" event={"ID":"da06502a-2382-4c5c-aa50-06912a1cadaf","Type":"ContainerStarted","Data":"600b1a40c866805546933934c70b6512c636c4dbbdad8cb2309abeb73e8706b9"}
Aug 04 18:28:00.928212 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.928193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-139.ec2.internal" event={"ID":"7d0a2e4e61221718b5319001dcdbfbbc","Type":"ContainerStarted","Data":"f322a0185caf19df678c7a21de71acc1fbb691f0d03cd1370459ac8f5c7bbed5"}
Aug 04 18:28:00.929459 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.929440    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd" event={"ID":"2ef3a825-5bff-4de4-8ce8-4b22a634770c","Type":"ContainerStarted","Data":"578a3ccc256fc69dbd42769bb2d7cebceb8b05fb95707bc26f479cb6069b60f6"}
Aug 04 18:28:00.933689 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.933666    2578 generic.go:356] "Generic (PLEG): container finished" podID="3f1e88cb-747e-47e7-a1fe-bfd764c13f39" containerID="a692e7a19a195c768c99cc0589c770433f0cb1b0622a7bc50a86d90770d76cca" exitCode=0
Aug 04 18:28:00.933787 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.933731    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wskjj" event={"ID":"3f1e88cb-747e-47e7-a1fe-bfd764c13f39","Type":"ContainerDied","Data":"a692e7a19a195c768c99cc0589c770433f0cb1b0622a7bc50a86d90770d76cca"}
Aug 04 18:28:00.935278 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.935234    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hwjp9" event={"ID":"05646d91-5551-4043-8542-b01200ff9771","Type":"ContainerStarted","Data":"10696b2b0f0eef8b52d42256f083ab7ee2d92f1f307e8eb36db68b58d4a187a9"}
Aug 04 18:28:00.938055 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.938033    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5" event={"ID":"853358c5-2d38-4e38-8d7b-d8ea471a92df","Type":"ContainerStarted","Data":"8e495dcb480bd08c8dd1673709eee295a94175e8ae5e57e445d5c964e0b4a55b"}
Aug 04 18:28:00.938133 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.938106    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5" event={"ID":"853358c5-2d38-4e38-8d7b-d8ea471a92df","Type":"ContainerStarted","Data":"87b996dd1e0bcd56f2e1a7db34f4bb18b74dc9e2a0d712c308158785d03a306c"}
Aug 04 18:28:00.938173 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.938152    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5" event={"ID":"853358c5-2d38-4e38-8d7b-d8ea471a92df","Type":"ContainerStarted","Data":"daecb0f6d32d2b143c7faf3d4e8c91124fb6543d414900a59b24555ab30b805e"}
Aug 04 18:28:00.938211 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.938194    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5" event={"ID":"853358c5-2d38-4e38-8d7b-d8ea471a92df","Type":"ContainerStarted","Data":"2531a856bbe845cf833b01823d60a7e93209e45aa150f32a39cdbc5f268d8f7d"}
Aug 04 18:28:00.938247 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.938230    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5" event={"ID":"853358c5-2d38-4e38-8d7b-d8ea471a92df","Type":"ContainerStarted","Data":"c6913d44a76ca887c69eb15b1fe30e13160548b57a36d7dfcbe2303cf6268f02"}
Aug 04 18:28:00.938289 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.938276    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5" event={"ID":"853358c5-2d38-4e38-8d7b-d8ea471a92df","Type":"ContainerStarted","Data":"cc4783958bc34dbe5140b542964cc59c7e345cdab96e23902ff2ebfdb402154b"}
Aug 04 18:28:00.939463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.939440    2578 generic.go:356] "Generic (PLEG): container finished" podID="a3e3b8db9dd2cfc3e4727a04a3b5eb0f" containerID="794ab351ca9b42ff1d79ab1813e5be399bda1763c8c5b463be604db8b5ef7c28" exitCode=0
Aug 04 18:28:00.939586 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.939506    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal" event={"ID":"a3e3b8db9dd2cfc3e4727a04a3b5eb0f","Type":"ContainerDied","Data":"794ab351ca9b42ff1d79ab1813e5be399bda1763c8c5b463be604db8b5ef7c28"}
Aug 04 18:28:00.940898 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.940878    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9g8kd" event={"ID":"274adda4-c5aa-448e-baf7-d4b48140929c","Type":"ContainerStarted","Data":"70f3610825c1b4cc162709d82cfc10707c0d8339f969050a7434bceb80070b67"}
Aug 04 18:28:00.961355 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:00.961302    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-486lc" podStartSLOduration=3.283881973 podStartE2EDuration="22.961287913s" totalImagesPullingTime="19.67740594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:00.960864182 +0000 UTC m=+22.827779306" watchObservedRunningTime="2026-08-04 18:28:00.961287913 +0000 UTC m=+22.828203033"
Aug 04 18:28:01.010381 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.010103    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8jqq2" podStartSLOduration=3.445582858 podStartE2EDuration="23.010083793s" totalImagesPullingTime="19.564500935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:01.009030796 +0000 UTC m=+22.875945897" watchObservedRunningTime="2026-08-04 18:28:01.010083793 +0000 UTC m=+22.876998900"
Aug 04 18:28:01.036717 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.036665    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9g8kd" podStartSLOduration=3.183905294 podStartE2EDuration="23.036650058s" totalImagesPullingTime="19.852744764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:01.036429888 +0000 UTC m=+22.903344989" watchObservedRunningTime="2026-08-04 18:28:01.036650058 +0000 UTC m=+22.903565159"
Aug 04 18:28:01.067377 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.067328    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-139.ec2.internal" podStartSLOduration=22.067312435 podStartE2EDuration="22.067312435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:27:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:01.066465536 +0000 UTC m=+22.933380637" watchObservedRunningTime="2026-08-04 18:28:01.067312435 +0000 UTC m=+22.934227535"
Aug 04 18:28:01.091023 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.090974    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xsv5g" podStartSLOduration=3.536944552 podStartE2EDuration="23.090960547s" totalImagesPullingTime="19.554015995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:01.090940706 +0000 UTC m=+22.957855807" watchObservedRunningTime="2026-08-04 18:28:01.090960547 +0000 UTC m=+22.957875648"
Aug 04 18:28:01.123950 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.123900    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hwjp9" podStartSLOduration=11.579043877 podStartE2EDuration="23.123883962s" totalImagesPullingTime="11.544840085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:01.122100261 +0000 UTC m=+22.989015373" watchObservedRunningTime="2026-08-04 18:28:01.123883962 +0000 UTC m=+22.990799063"
Aug 04 18:28:01.770251 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.770172    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:28:01.770983 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.770949    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:28:01.796146 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.796119    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:01.796312 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:01.796241    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:28:01.944092 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.944055    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal" event={"ID":"a3e3b8db9dd2cfc3e4727a04a3b5eb0f","Type":"ContainerStarted","Data":"c96df50b9aaea35d877ad4ab0248e0c482224204bf4432ec81cb1d0286096ae3"}
Aug 04 18:28:01.945481 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.945443    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ff5v6" event={"ID":"5391dbb5-02be-4374-b86d-ceda899618d3","Type":"ContainerStarted","Data":"806d96ffddd9512e3e3690bae1348cc68aca51a5cc0d0eff3770e0514fe46312"}
Aug 04 18:28:01.946251 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.946229    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:28:01.946628 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.946613    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hwjp9"
Aug 04 18:28:01.975957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.975919    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-139.ec2.internal" podStartSLOduration=22.975904785 podStartE2EDuration="22.975904785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:27:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:01.960323137 +0000 UTC m=+23.827238237" watchObservedRunningTime="2026-08-04 18:28:01.975904785 +0000 UTC m=+23.842819888"
Aug 04 18:28:01.993631 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:01.993563    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ff5v6" podStartSLOduration=3.141599718 podStartE2EDuration="22.993543996s" totalImagesPullingTime="19.851944278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:01.993120167 +0000 UTC m=+23.860035270" watchObservedRunningTime="2026-08-04 18:28:01.993543996 +0000 UTC m=+23.860459099"
Aug 04 18:28:02.264856 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:02.264823    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 04 18:28:02.629055 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:02.628952    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T18:28:02.264851245Z","UUID":"a54c11a2-19fb-4f49-893b-a5ab390c8e6d","Handler":null,"Name":"","Endpoint":""}
Aug 04 18:28:02.630659 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:02.630637    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 04 18:28:02.630802 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:02.630672    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 04 18:28:02.796567 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:02.796541    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:28:02.796720 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:02.796691    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:28:02.797335 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:02.796790    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:02.797335 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:02.796906    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:28:02.953209 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:02.953055    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5" event={"ID":"853358c5-2d38-4e38-8d7b-d8ea471a92df","Type":"ContainerStarted","Data":"9860e88fd1fa7a373b38c4473abb4e60a8df9f661ec0c7c2b5811221e90d9e2a"}
Aug 04 18:28:02.955938 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:02.955442    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd" event={"ID":"2ef3a825-5bff-4de4-8ce8-4b22a634770c","Type":"ContainerStarted","Data":"053dde2b89ab35b1b13aa7777ac5bf76ecfc639e5ab9e66ac9b0b164596925b8"}
Aug 04 18:28:03.796852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:03.796816    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:03.797016 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:03.796928    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:28:03.959053 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:03.958974    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd" event={"ID":"2ef3a825-5bff-4de4-8ce8-4b22a634770c","Type":"ContainerStarted","Data":"610c00bf9e918cb0b26f8d46ff612581fdd08cf22cb4862514aeda1e32c45d2e"}
Aug 04 18:28:03.983659 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:03.983611    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s2bxd" podStartSLOduration=3.945234301 podStartE2EDuration="25.983596301s" totalImagesPullingTime="22.038362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:27:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:03.982170359 +0000 UTC m=+25.849085460" watchObservedRunningTime="2026-08-04 18:28:03.983596301 +0000 UTC m=+25.850511416"
Aug 04 18:28:04.796484 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:04.796452    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:28:04.796684 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:04.796639    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:28:04.796756 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:04.796694    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:04.796816 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:04.796794    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:28:05.795909 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:05.795870    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:05.796305 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:05.795989    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:28:06.796208 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:06.796025    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:06.797153 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:06.796025    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:28:06.797153 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:06.796288    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:28:06.797153 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:06.796357    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:28:06.967269 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:06.967239    2578 generic.go:356] "Generic (PLEG): container finished" podID="3f1e88cb-747e-47e7-a1fe-bfd764c13f39" containerID="cd72e4e15e92fe6bc4e625d4fc96e5f9ee86cfe934bf7aefd24ec87b7606cc0a" exitCode=0
Aug 04 18:28:06.967432 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:06.967315    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wskjj" event={"ID":"3f1e88cb-747e-47e7-a1fe-bfd764c13f39","Type":"ContainerDied","Data":"cd72e4e15e92fe6bc4e625d4fc96e5f9ee86cfe934bf7aefd24ec87b7606cc0a"}
Aug 04 18:28:06.971091 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:06.971069    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5" event={"ID":"853358c5-2d38-4e38-8d7b-d8ea471a92df","Type":"ContainerStarted","Data":"4e8688a48f127489b5ff9544158b5529087005d3b99b5b5a8300710e739db47c"}
Aug 04 18:28:06.971330 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:06.971311    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:28:06.971412 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:06.971380    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:28:06.986740 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:06.986721    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:28:07.038715 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:07.038665    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5" podStartSLOduration=8.145428859999999 podStartE2EDuration="28.038649348s" totalImagesPullingTime="19.893220488s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:07.038285169 +0000 UTC m=+28.905200270" watchObservedRunningTime="2026-08-04 18:28:07.038649348 +0000 UTC m=+28.905564448"
Aug 04 18:28:07.796470 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:07.796438    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:07.796968 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:07.796563    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:28:07.974533 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:07.974314    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:28:07.989185 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:07.989159    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:28:08.461198 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:08.461112    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ds85h"]
Aug 04 18:28:08.461343 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:08.461232    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:08.461343 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:08.461321    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:28:08.465320 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:08.465294    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fcf2z"]
Aug 04 18:28:08.465456 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:08.465403    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:28:08.465505 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:08.465488    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:28:08.467850 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:08.467827    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h2rx8"]
Aug 04 18:28:08.467977 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:08.467928    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:08.468045 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:08.468025    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:28:08.978606 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:08.978443    2578 generic.go:356] "Generic (PLEG): container finished" podID="3f1e88cb-747e-47e7-a1fe-bfd764c13f39" containerID="d81e241a1c9abc1d59882748f7a9649e8043aaac45f27f27dec9958e41792b1c" exitCode=0
Aug 04 18:28:08.979012 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:08.978840    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wskjj" event={"ID":"3f1e88cb-747e-47e7-a1fe-bfd764c13f39","Type":"ContainerDied","Data":"d81e241a1c9abc1d59882748f7a9649e8043aaac45f27f27dec9958e41792b1c"}
Aug 04 18:28:09.796734 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:09.796705    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:09.796913 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:09.796746    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:09.796913 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:09.796806    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:28:09.796983 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:09.796926    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:28:10.796552 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:10.796516    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:28:10.796946 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:10.796660    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:28:10.985448 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:10.985371    2578 generic.go:356] "Generic (PLEG): container finished" podID="3f1e88cb-747e-47e7-a1fe-bfd764c13f39" containerID="530131ca982ee024eca9237d65325a392f04c1c64fa671e2793c198806551a8f" exitCode=0
Aug 04 18:28:10.985448 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:10.985422    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wskjj" event={"ID":"3f1e88cb-747e-47e7-a1fe-bfd764c13f39","Type":"ContainerDied","Data":"530131ca982ee024eca9237d65325a392f04c1c64fa671e2793c198806551a8f"}
Aug 04 18:28:11.308206 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:11.308162    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:28:11.308381 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:11.308311    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:28:11.308381 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:11.308379    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs podName:6ebbf61e-7a2b-472f-9232-8d9c81f96ded nodeName:}" failed. No retries permitted until 2026-08-04 18:28:43.308363549 +0000 UTC m=+65.175278628 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs") pod "network-metrics-daemon-fcf2z" (UID: "6ebbf61e-7a2b-472f-9232-8d9c81f96ded") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 18:28:11.409334 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:11.409287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:11.409499 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:11.409410    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 18:28:11.409499 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:11.409461    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret podName:d2ecaa08-30d0-47a3-b32f-50486a0462e2 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:43.40944891 +0000 UTC m=+65.276363990 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret") pod "global-pull-secret-syncer-h2rx8" (UID: "d2ecaa08-30d0-47a3-b32f-50486a0462e2") : object "kube-system"/"original-pull-secret" not registered
Aug 04 18:28:11.509680 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:11.509642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w2j2\" (UniqueName: \"kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2\") pod \"network-check-target-ds85h\" (UID: \"07c78662-4da8-47a9-b9ec-f03b7945ff62\") " pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:11.509849 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:11.509800    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 18:28:11.509849 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:11.509817    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 18:28:11.509849 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:11.509828    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9w2j2 for pod openshift-network-diagnostics/network-check-target-ds85h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:28:11.509949 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:11.509880    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2 podName:07c78662-4da8-47a9-b9ec-f03b7945ff62 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:43.509865593 +0000 UTC m=+65.376780671 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9w2j2" (UniqueName: "kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2") pod "network-check-target-ds85h" (UID: "07c78662-4da8-47a9-b9ec-f03b7945ff62") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 18:28:11.796097 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:11.796065    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:11.796257 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:11.796067    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:11.796295 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:11.796273    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h2rx8" podUID="d2ecaa08-30d0-47a3-b32f-50486a0462e2"
Aug 04 18:28:11.796331 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:11.796175    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds85h" podUID="07c78662-4da8-47a9-b9ec-f03b7945ff62"
Aug 04 18:28:12.796706 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:12.796681    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:28:12.797101 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:12.796789    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fcf2z" podUID="6ebbf61e-7a2b-472f-9232-8d9c81f96ded"
Aug 04 18:28:12.991459 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:12.991429    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-139.ec2.internal" event="NodeReady"
Aug 04 18:28:12.991617 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:12.991592    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 04 18:28:13.028780 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.028741    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm"]
Aug 04 18:28:13.033024 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.032983    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-575496654b-z8wsj"]
Aug 04 18:28:13.033551 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.033430    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm"
Aug 04 18:28:13.037055 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.037031    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"]
Aug 04 18:28:13.037289 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.037266    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.038590 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.038560    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Aug 04 18:28:13.038687 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.038642    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Aug 04 18:28:13.038687 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.038665    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-9z647\""
Aug 04 18:28:13.039135 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.039119    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Aug 04 18:28:13.039239 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.039140    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Aug 04 18:28:13.039698 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.039681    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 04 18:28:13.039782 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.039716    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 04 18:28:13.039782 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.039764    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-87ls6\""
Aug 04 18:28:13.039903 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.039887    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 04 18:28:13.041146 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.041128    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"]
Aug 04 18:28:13.041262 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.041247    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.044828 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.044810    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Aug 04 18:28:13.045325 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.045272    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:13.045819 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.045708    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Aug 04 18:28:13.046105 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.046086    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Aug 04 18:28:13.046360 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.046344    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Aug 04 18:28:13.046459 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.046437    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 04 18:28:13.047691 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.047608    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm"]
Aug 04 18:28:13.048560 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.048541    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jwfsp"]
Aug 04 18:28:13.048953 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.048935    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Aug 04 18:28:13.052478 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.052221    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.052855 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.052796    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"]
Aug 04 18:28:13.053494 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.053475    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"]
Aug 04 18:28:13.054093 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.054074    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-575496654b-z8wsj"]
Aug 04 18:28:13.055130 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.054670    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 04 18:28:13.055130 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.054820    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gbxls"]
Aug 04 18:28:13.055130 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.054986    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t8fxw\""
Aug 04 18:28:13.055343 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.055293    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 04 18:28:13.058456 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.058436    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbxls"
Aug 04 18:28:13.061178 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.061135    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 04 18:28:13.061358 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.061339    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 04 18:28:13.061668 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.061649    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ss77g\""
Aug 04 18:28:13.061862 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.061826    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 04 18:28:13.065520 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.065501    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jwfsp"]
Aug 04 18:28:13.067849 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.067825    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbxls"]
Aug 04 18:28:13.126992 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.124329    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f45pw\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-kube-api-access-f45pw\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.126992 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.124688    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aff1fa38-3505-4972-a71e-2a37e8c3fa23-ca-trust-extracted\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.126992 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.124803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-image-registry-private-configuration\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.126992 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.124838    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-bound-sa-token\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.126992 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.124903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-certificates\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.126992 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.124936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-trusted-ca\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.126992 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.124960    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f48809b8-78c4-48d7-9da3-eefe1d351636-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm\" (UID: \"f48809b8-78c4-48d7-9da3-eefe1d351636\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm"
Aug 04 18:28:13.126992 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.124990    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpvfd\" (UniqueName: \"kubernetes.io/projected/f48809b8-78c4-48d7-9da3-eefe1d351636-kube-api-access-mpvfd\") pod \"managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm\" (UID: \"f48809b8-78c4-48d7-9da3-eefe1d351636\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm"
Aug 04 18:28:13.126992 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.125026    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-installation-pull-secrets\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.126992 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.125122    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.225542 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225490    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-installation-pull-secrets\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.225542 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225540    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-hub\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.225796 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225561    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.225796 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225596    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.225796 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225614    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-ca\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.225796 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/452fb709-0018-44e1-b536-0ed6715b0d65-additional-service-ca\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.225796 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:13.225709    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:28:13.225796 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:13.225725    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575496654b-z8wsj: secret "image-registry-tls" not found
Aug 04 18:28:13.225796 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225744    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq9qp\" (UniqueName: \"kubernetes.io/projected/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-kube-api-access-wq9qp\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.225796 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:13.225789    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls podName:aff1fa38-3505-4972-a71e-2a37e8c3fa23 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:13.725768438 +0000 UTC m=+35.592683525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls") pod "image-registry-575496654b-z8wsj" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23") : secret "image-registry-tls" not found
Aug 04 18:28:13.226189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225806    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6-cert\") pod \"ingress-canary-gbxls\" (UID: \"61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6\") " pod="openshift-ingress-canary/ingress-canary-gbxls"
Aug 04 18:28:13.226189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67r2v\" (UniqueName: \"kubernetes.io/projected/240b0635-6109-4245-a7ea-a2107ac2efdc-kube-api-access-67r2v\") pod \"klusterlet-addon-workmgr-c976c4d6b-shshn\" (UID: \"240b0635-6109-4245-a7ea-a2107ac2efdc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:13.226189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.226189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f45pw\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-kube-api-access-f45pw\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.226189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225919    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/240b0635-6109-4245-a7ea-a2107ac2efdc-tmp\") pod \"klusterlet-addon-workmgr-c976c4d6b-shshn\" (UID: \"240b0635-6109-4245-a7ea-a2107ac2efdc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:13.226189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.225942    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-config-volume\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.226189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226034    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aff1fa38-3505-4972-a71e-2a37e8c3fa23-ca-trust-extracted\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.226189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226075    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.226189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmmtj\" (UniqueName: \"kubernetes.io/projected/61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6-kube-api-access-hmmtj\") pod \"ingress-canary-gbxls\" (UID: \"61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6\") " pod="openshift-ingress-canary/ingress-canary-gbxls"
Aug 04 18:28:13.226189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226177    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-image-registry-private-configuration\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.226682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-bound-sa-token\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.226682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226248    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfq4d\" (UniqueName: \"kubernetes.io/projected/452fb709-0018-44e1-b536-0ed6715b0d65-kube-api-access-hfq4d\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.226682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226279    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/240b0635-6109-4245-a7ea-a2107ac2efdc-klusterlet-config\") pod \"klusterlet-addon-workmgr-c976c4d6b-shshn\" (UID: \"240b0635-6109-4245-a7ea-a2107ac2efdc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:13.226682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226305    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-tmp-dir\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.226682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-certificates\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.226682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226408    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aff1fa38-3505-4972-a71e-2a37e8c3fa23-ca-trust-extracted\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.226682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226437    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-trusted-ca\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.226682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f48809b8-78c4-48d7-9da3-eefe1d351636-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm\" (UID: \"f48809b8-78c4-48d7-9da3-eefe1d351636\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm"
Aug 04 18:28:13.226682 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpvfd\" (UniqueName: \"kubernetes.io/projected/f48809b8-78c4-48d7-9da3-eefe1d351636-kube-api-access-mpvfd\") pod \"managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm\" (UID: \"f48809b8-78c4-48d7-9da3-eefe1d351636\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm"
Aug 04 18:28:13.227150 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.226945    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-certificates\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.227351 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.227329    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-trusted-ca\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.230821 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.230799    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-installation-pull-secrets\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.230921 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.230837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-image-registry-private-configuration\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.230921 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.230864    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f48809b8-78c4-48d7-9da3-eefe1d351636-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm\" (UID: \"f48809b8-78c4-48d7-9da3-eefe1d351636\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm"
Aug 04 18:28:13.239068 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.239035    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-bound-sa-token\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.239486 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.239466    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpvfd\" (UniqueName: \"kubernetes.io/projected/f48809b8-78c4-48d7-9da3-eefe1d351636-kube-api-access-mpvfd\") pod \"managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm\" (UID: \"f48809b8-78c4-48d7-9da3-eefe1d351636\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm"
Aug 04 18:28:13.239947 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.239929    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f45pw\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-kube-api-access-f45pw\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.261955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.261926    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:13.327961 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.327869    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.327961 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.327913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmmtj\" (UniqueName: \"kubernetes.io/projected/61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6-kube-api-access-hmmtj\") pod \"ingress-canary-gbxls\" (UID: \"61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6\") " pod="openshift-ingress-canary/ingress-canary-gbxls"
Aug 04 18:28:13.327961 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.327942    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfq4d\" (UniqueName: \"kubernetes.io/projected/452fb709-0018-44e1-b536-0ed6715b0d65-kube-api-access-hfq4d\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.328240 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.327972    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/240b0635-6109-4245-a7ea-a2107ac2efdc-klusterlet-config\") pod \"klusterlet-addon-workmgr-c976c4d6b-shshn\" (UID: \"240b0635-6109-4245-a7ea-a2107ac2efdc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:13.328240 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.327998    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-tmp-dir\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.328240 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.328198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-hub\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.328359 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.328238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.328359 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.328284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-ca\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.328359 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.328313    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/452fb709-0018-44e1-b536-0ed6715b0d65-additional-service-ca\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.328359 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.328337    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-tmp-dir\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.328359 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.328340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq9qp\" (UniqueName: \"kubernetes.io/projected/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-kube-api-access-wq9qp\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.328626 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.328378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6-cert\") pod \"ingress-canary-gbxls\" (UID: \"61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6\") " pod="openshift-ingress-canary/ingress-canary-gbxls"
Aug 04 18:28:13.328626 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.328407    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67r2v\" (UniqueName: \"kubernetes.io/projected/240b0635-6109-4245-a7ea-a2107ac2efdc-kube-api-access-67r2v\") pod \"klusterlet-addon-workmgr-c976c4d6b-shshn\" (UID: \"240b0635-6109-4245-a7ea-a2107ac2efdc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:13.328626 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.328433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.328626 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.328462    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/240b0635-6109-4245-a7ea-a2107ac2efdc-tmp\") pod \"klusterlet-addon-workmgr-c976c4d6b-shshn\" (UID: \"240b0635-6109-4245-a7ea-a2107ac2efdc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:13.328626 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.328486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-config-volume\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.328897 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:13.328879    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:28:13.328971 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:13.328956    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls podName:1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:13.8289364 +0000 UTC m=+35.695851495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls") pod "dns-default-jwfsp" (UID: "1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11") : secret "dns-default-metrics-tls" not found
Aug 04 18:28:13.329269 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.329055    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-config-volume\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.329269 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.329217    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/452fb709-0018-44e1-b536-0ed6715b0d65-additional-service-ca\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.329422 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.329406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/240b0635-6109-4245-a7ea-a2107ac2efdc-tmp\") pod \"klusterlet-addon-workmgr-c976c4d6b-shshn\" (UID: \"240b0635-6109-4245-a7ea-a2107ac2efdc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:13.331341 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.331320    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.332242 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.332191    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-hub\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.332354 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.332331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-ca\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.332417 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.332391    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/240b0635-6109-4245-a7ea-a2107ac2efdc-klusterlet-config\") pod \"klusterlet-addon-workmgr-c976c4d6b-shshn\" (UID: \"240b0635-6109-4245-a7ea-a2107ac2efdc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:13.332475 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.332439    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/452fb709-0018-44e1-b536-0ed6715b0d65-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.332890 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.332862    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6-cert\") pod \"ingress-canary-gbxls\" (UID: \"61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6\") " pod="openshift-ingress-canary/ingress-canary-gbxls"
Aug 04 18:28:13.337420 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.337386    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfq4d\" (UniqueName: \"kubernetes.io/projected/452fb709-0018-44e1-b536-0ed6715b0d65-kube-api-access-hfq4d\") pod \"cluster-proxy-proxy-agent-695cf4fb46-b4lhc\" (UID: \"452fb709-0018-44e1-b536-0ed6715b0d65\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.338025 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.338003    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmmtj\" (UniqueName: \"kubernetes.io/projected/61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6-kube-api-access-hmmtj\") pod \"ingress-canary-gbxls\" (UID: \"61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6\") " pod="openshift-ingress-canary/ingress-canary-gbxls"
Aug 04 18:28:13.338241 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.338218    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67r2v\" (UniqueName: \"kubernetes.io/projected/240b0635-6109-4245-a7ea-a2107ac2efdc-kube-api-access-67r2v\") pod \"klusterlet-addon-workmgr-c976c4d6b-shshn\" (UID: \"240b0635-6109-4245-a7ea-a2107ac2efdc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:13.339302 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.339275    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq9qp\" (UniqueName: \"kubernetes.io/projected/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-kube-api-access-wq9qp\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.356314 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.356281    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm"
Aug 04 18:28:13.373104 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.373076    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:13.384686 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.384647    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:28:13.391407 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.391378    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:13.405566 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.405513    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbxls"
Aug 04 18:28:13.534227 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.534155    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm"]
Aug 04 18:28:13.541547 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:28:13.541506    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf48809b8_78c4_48d7_9da3_eefe1d351636.slice/crio-78490145a08da3d2fc2ebc7aab317b6470e3bac3baeb4152e96c3c4c18b1f13a WatchSource:0}: Error finding container 78490145a08da3d2fc2ebc7aab317b6470e3bac3baeb4152e96c3c4c18b1f13a: Status 404 returned error can't find the container with id 78490145a08da3d2fc2ebc7aab317b6470e3bac3baeb4152e96c3c4c18b1f13a
Aug 04 18:28:13.569498 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.569465    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"]
Aug 04 18:28:13.572643 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.572563    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"]
Aug 04 18:28:13.575699 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:28:13.575349    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod452fb709_0018_44e1_b536_0ed6715b0d65.slice/crio-202bdd3355f03b7377819a08f452ca2f0640ffe6d51bd761f092fceb82d5b0b4 WatchSource:0}: Error finding container 202bdd3355f03b7377819a08f452ca2f0640ffe6d51bd761f092fceb82d5b0b4: Status 404 returned error can't find the container with id 202bdd3355f03b7377819a08f452ca2f0640ffe6d51bd761f092fceb82d5b0b4
Aug 04 18:28:13.598440 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.598400    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbxls"]
Aug 04 18:28:13.602313 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:28:13.602287    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61245fc9_3d68_4d5b_a57e_3f3c3bc9b6b6.slice/crio-1210b9c8c312b9186d6004ba5f7727d61e92cff40ecfebaed7fb87afbf669d39 WatchSource:0}: Error finding container 1210b9c8c312b9186d6004ba5f7727d61e92cff40ecfebaed7fb87afbf669d39: Status 404 returned error can't find the container with id 1210b9c8c312b9186d6004ba5f7727d61e92cff40ecfebaed7fb87afbf669d39
Aug 04 18:28:13.731973 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.731937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:13.732156 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:13.732114    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:28:13.732156 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:13.732135    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575496654b-z8wsj: secret "image-registry-tls" not found
Aug 04 18:28:13.732275 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:13.732213    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls podName:aff1fa38-3505-4972-a71e-2a37e8c3fa23 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:14.732191338 +0000 UTC m=+36.599106439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls") pod "image-registry-575496654b-z8wsj" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23") : secret "image-registry-tls" not found
Aug 04 18:28:13.796732 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.796697    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:13.796732 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.796726    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:13.799689 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.799661    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bpqnp\""
Aug 04 18:28:13.799689 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.799687    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 18:28:13.799864 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.799661    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 18:28:13.799864 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.799661    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 18:28:13.833134 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.833047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:13.833298 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:13.833183    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:28:13.833298 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:13.833245    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls podName:1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:14.833228361 +0000 UTC m=+36.700143443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls") pod "dns-default-jwfsp" (UID: "1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11") : secret "dns-default-metrics-tls" not found
Aug 04 18:28:13.993716 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.993677    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn" event={"ID":"240b0635-6109-4245-a7ea-a2107ac2efdc","Type":"ContainerStarted","Data":"2199f7a021aa63df26efad7d3f7b59cfc83e850c0c8b04d0ab3ecf149a2cd14b"}
Aug 04 18:28:13.994749 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.994729    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm" event={"ID":"f48809b8-78c4-48d7-9da3-eefe1d351636","Type":"ContainerStarted","Data":"78490145a08da3d2fc2ebc7aab317b6470e3bac3baeb4152e96c3c4c18b1f13a"}
Aug 04 18:28:13.995791 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.995756    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbxls" event={"ID":"61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6","Type":"ContainerStarted","Data":"1210b9c8c312b9186d6004ba5f7727d61e92cff40ecfebaed7fb87afbf669d39"}
Aug 04 18:28:13.996842 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:13.996815    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" event={"ID":"452fb709-0018-44e1-b536-0ed6715b0d65","Type":"ContainerStarted","Data":"202bdd3355f03b7377819a08f452ca2f0640ffe6d51bd761f092fceb82d5b0b4"}
Aug 04 18:28:14.742073 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:14.741450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:14.742073 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:14.741646    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:28:14.742073 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:14.741663    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575496654b-z8wsj: secret "image-registry-tls" not found
Aug 04 18:28:14.742073 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:14.741722    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls podName:aff1fa38-3505-4972-a71e-2a37e8c3fa23 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:16.741703638 +0000 UTC m=+38.608618734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls") pod "image-registry-575496654b-z8wsj" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23") : secret "image-registry-tls" not found
Aug 04 18:28:14.800107 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:14.799301    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:28:14.803778 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:14.803752    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 18:28:14.812996 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:14.804274    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zw2lq\""
Aug 04 18:28:14.842501 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:14.842464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:14.842705 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:14.842654    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:28:14.842782 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:14.842718    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls podName:1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:16.842699096 +0000 UTC m=+38.709614181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls") pod "dns-default-jwfsp" (UID: "1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11") : secret "dns-default-metrics-tls" not found
Aug 04 18:28:16.763707 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:16.763308    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:16.763707 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:16.763683    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:28:16.763707 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:16.763702    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575496654b-z8wsj: secret "image-registry-tls" not found
Aug 04 18:28:16.764288 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:16.763767    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls podName:aff1fa38-3505-4972-a71e-2a37e8c3fa23 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:20.763749976 +0000 UTC m=+42.630665073 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls") pod "image-registry-575496654b-z8wsj" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23") : secret "image-registry-tls" not found
Aug 04 18:28:16.864335 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:16.864288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:16.864968 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:16.864493    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:28:16.864968 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:16.864600    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls podName:1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:20.864560276 +0000 UTC m=+42.731475360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls") pod "dns-default-jwfsp" (UID: "1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11") : secret "dns-default-metrics-tls" not found
Aug 04 18:28:17.469031 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:17.469000    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:17.531954 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:17.531921    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:18.017700 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:18.017653    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:20.792200 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:20.792166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:20.792728 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:20.792300    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:28:20.792728 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:20.792314    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575496654b-z8wsj: secret "image-registry-tls" not found
Aug 04 18:28:20.792728 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:20.792369    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls podName:aff1fa38-3505-4972-a71e-2a37e8c3fa23 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:28.792352593 +0000 UTC m=+50.659267690 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls") pod "image-registry-575496654b-z8wsj" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23") : secret "image-registry-tls" not found
Aug 04 18:28:20.893238 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:20.893200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:20.893429 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:20.893360    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:28:20.893429 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:20.893423    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls podName:1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:28.893406133 +0000 UTC m=+50.760321212 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls") pod "dns-default-jwfsp" (UID: "1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11") : secret "dns-default-metrics-tls" not found
Aug 04 18:28:22.182301 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:22.182264    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:22.252269 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:22.252230    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:25.030162 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.030077    2578 generic.go:356] "Generic (PLEG): container finished" podID="3f1e88cb-747e-47e7-a1fe-bfd764c13f39" containerID="a20ddf56272d8a88b03b2013bfd97261601060f2fc02e2bac8a1f0494e363061" exitCode=0
Aug 04 18:28:25.030162 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.030136    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wskjj" event={"ID":"3f1e88cb-747e-47e7-a1fe-bfd764c13f39","Type":"ContainerDied","Data":"a20ddf56272d8a88b03b2013bfd97261601060f2fc02e2bac8a1f0494e363061"}
Aug 04 18:28:25.031694 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.031666    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn" event={"ID":"240b0635-6109-4245-a7ea-a2107ac2efdc","Type":"ContainerStarted","Data":"5cc3889688b4b733df42447d51e2d1a87e966d17d665ceb1d0724f439bb7fa5e"}
Aug 04 18:28:25.031824 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.031810    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:25.033354 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.033329    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm" event={"ID":"f48809b8-78c4-48d7-9da3-eefe1d351636","Type":"ContainerStarted","Data":"93a6d1a23e8c47d4fd48d0c39148c1aee6a2e4ad59a505bdcd16adcfd918f4bf"}
Aug 04 18:28:25.033867 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.033841    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn"
Aug 04 18:28:25.034909 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.034884    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbxls" event={"ID":"61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6","Type":"ContainerStarted","Data":"fd704cc2d338a33b1ab20b1b23749641e781e53bc4ae35d7c40cf15b23e8db9c"}
Aug 04 18:28:25.036951 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.036932    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" event={"ID":"452fb709-0018-44e1-b536-0ed6715b0d65","Type":"ContainerStarted","Data":"703fe970b87035ff47fc8a7977d7a268aac12b84b143b3b55dee00211e1ee667"}
Aug 04 18:28:25.037045 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.036960    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" event={"ID":"452fb709-0018-44e1-b536-0ed6715b0d65","Type":"ContainerStarted","Data":"80e0af12151acd3929900ead16b9b2335aea998a0fd3e0642ec2d3a23cfb2f0d"}
Aug 04 18:28:25.037045 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.036972    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" event={"ID":"452fb709-0018-44e1-b536-0ed6715b0d65","Type":"ContainerStarted","Data":"7a4159f82e98c16e8ea745b618975c99ab8a695dd33a5eaa8c19f9c613de7135"}
Aug 04 18:28:25.102900 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.102852    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gbxls" podStartSLOduration=1.14749137 podStartE2EDuration="12.102837175s" totalImagesPullingTime="10.955345805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:28:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:25.102695454 +0000 UTC m=+46.969610554" watchObservedRunningTime="2026-08-04 18:28:25.102837175 +0000 UTC m=+46.969752276"
Aug 04 18:28:25.126233 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.126156    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7889b9dcd6-mhxsm" podStartSLOduration=27.102697367 podStartE2EDuration="38.126133382s" totalImagesPullingTime="11.023436015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:25.125322943 +0000 UTC m=+46.992238041" watchObservedRunningTime="2026-08-04 18:28:25.126133382 +0000 UTC m=+46.993048483"
Aug 04 18:28:25.145805 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.145590    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-c976c4d6b-shshn" podStartSLOduration=27.162787462 podStartE2EDuration="38.14555381s" totalImagesPullingTime="10.982766348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:25.144842715 +0000 UTC m=+47.011757816" watchObservedRunningTime="2026-08-04 18:28:25.14555381 +0000 UTC m=+47.012468896"
Aug 04 18:28:25.168281 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:25.168221    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" podStartSLOduration=27.186562518 podStartE2EDuration="38.168204223s" totalImagesPullingTime="10.981641705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:25.167481469 +0000 UTC m=+47.034396583" watchObservedRunningTime="2026-08-04 18:28:25.168204223 +0000 UTC m=+47.035119301"
Aug 04 18:28:26.042997 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:26.042965    2578 generic.go:356] "Generic (PLEG): container finished" podID="3f1e88cb-747e-47e7-a1fe-bfd764c13f39" containerID="dbe01d16da18bd6bfdcb044ebf9eaca7e80d4e591e30dd97d9ec197f74d50525" exitCode=0
Aug 04 18:28:26.043457 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:26.043051    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wskjj" event={"ID":"3f1e88cb-747e-47e7-a1fe-bfd764c13f39","Type":"ContainerDied","Data":"dbe01d16da18bd6bfdcb044ebf9eaca7e80d4e591e30dd97d9ec197f74d50525"}
Aug 04 18:28:27.049717 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:27.049675    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wskjj" event={"ID":"3f1e88cb-747e-47e7-a1fe-bfd764c13f39","Type":"ContainerStarted","Data":"3737ffa58f9639d1cf5232cc2956f668b3fa5dab1fe9d4e2d32d6ad1e9c21a59"}
Aug 04 18:28:27.074922 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:27.074872    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wskjj" podStartSLOduration=7.965907115 podStartE2EDuration="49.07485431s" totalImagesPullingTime="41.108947195s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:27:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:27.072959469 +0000 UTC m=+48.939874573" watchObservedRunningTime="2026-08-04 18:28:27.07485431 +0000 UTC m=+48.941769440"
Aug 04 18:28:27.200291 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:27.200257    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:27.317437 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:27.317351    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:28.858085 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:28.858043    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:28.858475 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:28.858194    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:28:28.858475 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:28.858211    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575496654b-z8wsj: secret "image-registry-tls" not found
Aug 04 18:28:28.858475 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:28.858276    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls podName:aff1fa38-3505-4972-a71e-2a37e8c3fa23 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:44.858260764 +0000 UTC m=+66.725175843 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls") pod "image-registry-575496654b-z8wsj" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23") : secret "image-registry-tls" not found
Aug 04 18:28:28.959361 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:28.959315    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:28.959525 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:28.959464    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:28:28.959564 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:28.959540    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls podName:1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11 nodeName:}" failed. No retries permitted until 2026-08-04 18:28:44.959519054 +0000 UTC m=+66.826434136 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls") pod "dns-default-jwfsp" (UID: "1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11") : secret "dns-default-metrics-tls" not found
Aug 04 18:28:30.827047 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:30.827011    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:30.888003 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:30.887975    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:31.827852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:31.827810    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:31.927892 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:31.927855    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:34.498388 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:34.498336    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:38.345305 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:38.345268    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:38.405192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:38.405163    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:39.992149 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:39.992114    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mfgv5"
Aug 04 18:28:40.152631 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:40.152599    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:40.259899 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:40.259812    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:40.380219 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:40.380186    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:41.134538 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:41.134506    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:43.371904 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.371862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:28:43.374679 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.374661    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 18:28:43.383011 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:43.382990    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 04 18:28:43.383082 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:43.383064    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs podName:6ebbf61e-7a2b-472f-9232-8d9c81f96ded nodeName:}" failed. No retries permitted until 2026-08-04 18:29:47.383041396 +0000 UTC m=+129.249956476 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs") pod "network-metrics-daemon-fcf2z" (UID: "6ebbf61e-7a2b-472f-9232-8d9c81f96ded") : secret "metrics-daemon-secret" not found
Aug 04 18:28:43.472721 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.472670    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:43.475853 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.475833    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 18:28:43.486107 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.486075    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2ecaa08-30d0-47a3-b32f-50486a0462e2-original-pull-secret\") pod \"global-pull-secret-syncer-h2rx8\" (UID: \"d2ecaa08-30d0-47a3-b32f-50486a0462e2\") " pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:43.510222 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.510186    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h2rx8"
Aug 04 18:28:43.573672 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.573604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w2j2\" (UniqueName: \"kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2\") pod \"network-check-target-ds85h\" (UID: \"07c78662-4da8-47a9-b9ec-f03b7945ff62\") " pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:43.576954 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.576931    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 18:28:43.587652 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.587620    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 18:28:43.597280 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.597251    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w2j2\" (UniqueName: \"kubernetes.io/projected/07c78662-4da8-47a9-b9ec-f03b7945ff62-kube-api-access-9w2j2\") pod \"network-check-target-ds85h\" (UID: \"07c78662-4da8-47a9-b9ec-f03b7945ff62\") " pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:43.644137 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.644054    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h2rx8"]
Aug 04 18:28:43.647598 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:28:43.647550    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2ecaa08_30d0_47a3_b32f_50486a0462e2.slice/crio-fd9448bb6f412f9afc39bce889ee315e7b06c1fd481b33032ecda89ecbe4cc9a WatchSource:0}: Error finding container fd9448bb6f412f9afc39bce889ee315e7b06c1fd481b33032ecda89ecbe4cc9a: Status 404 returned error can't find the container with id fd9448bb6f412f9afc39bce889ee315e7b06c1fd481b33032ecda89ecbe4cc9a
Aug 04 18:28:43.820226 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.820191    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bpqnp\""
Aug 04 18:28:43.828374 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.828325    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:43.948252 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:43.948220    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ds85h"]
Aug 04 18:28:43.951203 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:28:43.951178    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07c78662_4da8_47a9_b9ec_f03b7945ff62.slice/crio-7bc4898851bf89cff847d2024f39b7dbedc8fb653e73ab91c62bdd26572b8e8d WatchSource:0}: Error finding container 7bc4898851bf89cff847d2024f39b7dbedc8fb653e73ab91c62bdd26572b8e8d: Status 404 returned error can't find the container with id 7bc4898851bf89cff847d2024f39b7dbedc8fb653e73ab91c62bdd26572b8e8d
Aug 04 18:28:44.074442 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:44.074395    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:44.101515 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:44.101475    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ds85h" event={"ID":"07c78662-4da8-47a9-b9ec-f03b7945ff62","Type":"ContainerStarted","Data":"7bc4898851bf89cff847d2024f39b7dbedc8fb653e73ab91c62bdd26572b8e8d"}
Aug 04 18:28:44.102566 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:44.102545    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h2rx8" event={"ID":"d2ecaa08-30d0-47a3-b32f-50486a0462e2","Type":"ContainerStarted","Data":"fd9448bb6f412f9afc39bce889ee315e7b06c1fd481b33032ecda89ecbe4cc9a"}
Aug 04 18:28:44.130914 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:44.130875    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:44.511497 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:44.511439    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:44.566570 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:44.566536    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:44.885726 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:44.885686    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:28:44.885919 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:44.885858    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:28:44.885919 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:44.885880    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575496654b-z8wsj: secret "image-registry-tls" not found
Aug 04 18:28:44.886023 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:44.885950    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls podName:aff1fa38-3505-4972-a71e-2a37e8c3fa23 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:16.885928477 +0000 UTC m=+98.752843568 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls") pod "image-registry-575496654b-z8wsj" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23") : secret "image-registry-tls" not found
Aug 04 18:28:44.987072 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:44.987034    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:28:44.987245 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:44.987195    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:28:44.987312 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:28:44.987270    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls podName:1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:16.987247329 +0000 UTC m=+98.854162653 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls") pod "dns-default-jwfsp" (UID: "1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11") : secret "dns-default-metrics-tls" not found
Aug 04 18:28:47.953340 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:47.953293    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:48.050967 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:48.050927    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:48.527999 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:48.527957    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:48.605990 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:48.605958    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:28:49.122126 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:49.122100    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ds85h" event={"ID":"07c78662-4da8-47a9-b9ec-f03b7945ff62","Type":"ContainerStarted","Data":"a15af70eb22ac9c8aaaecb263d0a6615f7a881f00cbb3834d04b0beb7cdeaaf7"}
Aug 04 18:28:49.122466 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:49.122330    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:28:49.139855 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:49.139805    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ds85h" podStartSLOduration=65.047457148 podStartE2EDuration="1m10.13979006s" totalImagesPullingTime="5.092332912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:49.139006791 +0000 UTC m=+71.005921894" watchObservedRunningTime="2026-08-04 18:28:49.13979006 +0000 UTC m=+71.006705161"
Aug 04 18:28:50.126318 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:50.126283    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h2rx8" event={"ID":"d2ecaa08-30d0-47a3-b32f-50486a0462e2","Type":"ContainerStarted","Data":"553b4c1c2a552b8d9be885719a0b4a678fc51bde7e7283f866b220d514a8b5e9"}
Aug 04 18:28:50.143149 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:28:50.143100    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h2rx8" podStartSLOduration=65.741724409 podStartE2EDuration="1m11.1430853s" totalImagesPullingTime="5.401360891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:28:50.142476264 +0000 UTC m=+72.009391366" watchObservedRunningTime="2026-08-04 18:28:50.1430853 +0000 UTC m=+72.010000400"
Aug 04 18:29:01.270611 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:01.270470    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:01.369390 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:01.369359    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:08.910456 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:08.910417    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:08.979504 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:08.979472    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:10.484077 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:10.484046    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:10.552444 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:10.552415    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:13.348225 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:13.348187    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:13.413497 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:13.413463    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:15.248352 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:15.248306    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:15.350071 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:15.350042    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:15.475178 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:15.475148    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:15.565196 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:15.565164    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:15.634062 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:15.634030    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:15.995189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:15.995080    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:16.000815 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.000789    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zgvv8"]
Aug 04 18:29:16.003547 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.003526    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.006716 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.006685    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 04 18:29:16.006716 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.006710    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 04 18:29:16.007293 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.007276    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 04 18:29:16.007382 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.007359    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 04 18:29:16.007956 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.007931    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hcmth\""
Aug 04 18:29:16.019485 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.019460    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zgvv8"]
Aug 04 18:29:16.091137 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.091101    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:16.128420 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.128372    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9852\" (UniqueName: \"kubernetes.io/projected/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-api-access-n9852\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.128553 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.128433    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.128553 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.128509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/65dfd213-63dd-44d3-9803-9b33b337ac19-data-volume\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.128553 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.128544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.128689 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.128651    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/65dfd213-63dd-44d3-9803-9b33b337ac19-crio-socket\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.229626 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.229590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/65dfd213-63dd-44d3-9803-9b33b337ac19-crio-socket\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.229759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.229650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9852\" (UniqueName: \"kubernetes.io/projected/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-api-access-n9852\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.229759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.229673    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.229759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.229689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/65dfd213-63dd-44d3-9803-9b33b337ac19-data-volume\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.229759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.229705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/65dfd213-63dd-44d3-9803-9b33b337ac19-crio-socket\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.229759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.229716    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.229936 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:16.229792    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 18:29:16.229936 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:16.229809    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 18:29:16.229936 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:16.229866    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm podName:65dfd213-63dd-44d3-9803-9b33b337ac19 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:16.729850891 +0000 UTC m=+98.596765970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgvv8" (UID: "65dfd213-63dd-44d3-9803-9b33b337ac19") : configmap "kube-rbac-proxy" not found
Aug 04 18:29:16.229936 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:16.229881    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls podName:65dfd213-63dd-44d3-9803-9b33b337ac19 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:16.729873347 +0000 UTC m=+98.596788426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgvv8" (UID: "65dfd213-63dd-44d3-9803-9b33b337ac19") : secret "insights-runtime-extractor-tls" not found
Aug 04 18:29:16.230083 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.230042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/65dfd213-63dd-44d3-9803-9b33b337ac19-data-volume\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.241286 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.241254    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9852\" (UniqueName: \"kubernetes.io/projected/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-api-access-n9852\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.734427 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.734371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.734967 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:16.734500    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 18:29:16.734967 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.734510    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:16.734967 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:16.734568    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm podName:65dfd213-63dd-44d3-9803-9b33b337ac19 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:17.734549565 +0000 UTC m=+99.601464659 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgvv8" (UID: "65dfd213-63dd-44d3-9803-9b33b337ac19") : configmap "kube-rbac-proxy" not found
Aug 04 18:29:16.734967 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:16.734684    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 18:29:16.734967 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:16.734738    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls podName:65dfd213-63dd-44d3-9803-9b33b337ac19 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:17.734723651 +0000 UTC m=+99.601638752 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgvv8" (UID: "65dfd213-63dd-44d3-9803-9b33b337ac19") : secret "insights-runtime-extractor-tls" not found
Aug 04 18:29:16.936098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:16.936059    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls\") pod \"image-registry-575496654b-z8wsj\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") " pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:29:16.936283 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:16.936218    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Aug 04 18:29:16.936283 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:16.936237    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-575496654b-z8wsj: secret "image-registry-tls" not found
Aug 04 18:29:16.936351 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:16.936298    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls podName:aff1fa38-3505-4972-a71e-2a37e8c3fa23 nodeName:}" failed. No retries permitted until 2026-08-04 18:30:20.936279584 +0000 UTC m=+162.803194679 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls") pod "image-registry-575496654b-z8wsj" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23") : secret "image-registry-tls" not found
Aug 04 18:29:17.036848 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:17.036816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:29:17.036996 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:17.036976    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 18:29:17.037066 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:17.037055    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls podName:1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11 nodeName:}" failed. No retries permitted until 2026-08-04 18:30:21.037038754 +0000 UTC m=+162.903953834 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls") pod "dns-default-jwfsp" (UID: "1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11") : secret "dns-default-metrics-tls" not found
Aug 04 18:29:17.656186 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:17.656153    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:17.743201 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:17.743167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:17.743612 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:17.743223    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:17.743612 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:17.743282    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 18:29:17.743612 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:17.743318    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 18:29:17.743612 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:17.743330    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm podName:65dfd213-63dd-44d3-9803-9b33b337ac19 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:19.743315205 +0000 UTC m=+101.610230284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgvv8" (UID: "65dfd213-63dd-44d3-9803-9b33b337ac19") : configmap "kube-rbac-proxy" not found
Aug 04 18:29:17.743612 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:17.743480    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls podName:65dfd213-63dd-44d3-9803-9b33b337ac19 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:19.743455309 +0000 UTC m=+101.610370397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgvv8" (UID: "65dfd213-63dd-44d3-9803-9b33b337ac19") : secret "insights-runtime-extractor-tls" not found
Aug 04 18:29:17.759000 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:17.758969    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:18.548566 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:18.548533    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8jqq2_da06502a-2382-4c5c-aa50-06912a1cadaf/dns-node-resolver/0.log"
Aug 04 18:29:18.594457 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:18.594422    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:18.907863 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:18.907774    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:19.018837 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:19.018799    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:19.131644 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:19.131609    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:19.232494 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:19.232387    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:19.552722 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:19.552699    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9g8kd_274adda4-c5aa-448e-baf7-d4b48140929c/node-ca/0.log"
Aug 04 18:29:19.762248 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:19.762215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:19.762467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:19.762259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:19.762467 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:19.762392    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 18:29:19.762467 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:19.762457    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 18:29:19.762669 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:19.762525    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls podName:65dfd213-63dd-44d3-9803-9b33b337ac19 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:23.762502292 +0000 UTC m=+105.629417391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgvv8" (UID: "65dfd213-63dd-44d3-9803-9b33b337ac19") : secret "insights-runtime-extractor-tls" not found
Aug 04 18:29:19.762669 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:19.762546    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm podName:65dfd213-63dd-44d3-9803-9b33b337ac19 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:23.762536211 +0000 UTC m=+105.629451298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgvv8" (UID: "65dfd213-63dd-44d3-9803-9b33b337ac19") : configmap "kube-rbac-proxy" not found
Aug 04 18:29:19.786643 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:19.786603    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:20.129021 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:20.128990    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ds85h"
Aug 04 18:29:20.407311 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:20.407224    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:20.545367 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:20.545340    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbxls_61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6/serve-healthcheck-canary/0.log"
Aug 04 18:29:20.790871 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:20.790841    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:21.203211 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:21.203135    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:21.587570 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:21.587531    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:22.000303 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:22.000224    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:22.174445 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:22.174412    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:22.243595 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:22.243544    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:22.778104 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:22.778075    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:22.977653 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:22.977626    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:23.421454 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:23.421413    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:23.605773 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:23.605741    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:23.688904 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:23.688828    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:23.792770 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:23.792724    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:23.792946 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:23.792789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:23.792946 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:23.792868    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 18:29:23.792946 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:23.792912    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 18:29:23.792946 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:23.792932    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls podName:65dfd213-63dd-44d3-9803-9b33b337ac19 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:31.792916569 +0000 UTC m=+113.659831649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgvv8" (UID: "65dfd213-63dd-44d3-9803-9b33b337ac19") : secret "insights-runtime-extractor-tls" not found
Aug 04 18:29:23.793082 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:23.792963    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm podName:65dfd213-63dd-44d3-9803-9b33b337ac19 nodeName:}" failed. No retries permitted until 2026-08-04 18:29:31.792948184 +0000 UTC m=+113.659863267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgvv8" (UID: "65dfd213-63dd-44d3-9803-9b33b337ac19") : configmap "kube-rbac-proxy" not found
Aug 04 18:29:23.998617 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:23.998522    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:24.192670 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:24.192636    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:24.391521 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:24.391492    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:25.288833 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:25.288792    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:25.388742 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:25.388708    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:25.594634 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:25.594531    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:25.801990 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:25.801962    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:26.245802 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:26.245771    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:26.791041 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:26.791003    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:27.188697 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:27.188616    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:27.395402 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:27.395366    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:28.001978 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:28.001949    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:28.200536 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:28.200502    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:28.605840 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:28.605809    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:28.949796 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:28.949714    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:29.064119 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:29.064091    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:29.192908 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:29.192872    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:29.495412 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:29.495380    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:29.797776 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:29.797746    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:29.995138 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:29.995108    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:31.854943 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:31.854902    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:31.855337 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:31.854993    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:31.855484 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:31.855458    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/65dfd213-63dd-44d3-9803-9b33b337ac19-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:31.857234 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:31.857217    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/65dfd213-63dd-44d3-9803-9b33b337ac19-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgvv8\" (UID: \"65dfd213-63dd-44d3-9803-9b33b337ac19\") " pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:31.912312 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:31.912260    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zgvv8"
Aug 04 18:29:32.038975 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:32.038940    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zgvv8"]
Aug 04 18:29:32.042033 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:29:32.042006    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65dfd213_63dd_44d3_9803_9b33b337ac19.slice/crio-aa5024dadd33dbd75ecee4b18b3613199e5b9ac0518572483255791fc41fa8b1 WatchSource:0}: Error finding container aa5024dadd33dbd75ecee4b18b3613199e5b9ac0518572483255791fc41fa8b1: Status 404 returned error can't find the container with id aa5024dadd33dbd75ecee4b18b3613199e5b9ac0518572483255791fc41fa8b1
Aug 04 18:29:32.248906 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:32.248823    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgvv8" event={"ID":"65dfd213-63dd-44d3-9803-9b33b337ac19","Type":"ContainerStarted","Data":"a5e9bf1790100859d50b1daecbaf0079678193c17ca45cc1f961d3ee8504b692"}
Aug 04 18:29:32.248906 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:32.248877    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgvv8" event={"ID":"65dfd213-63dd-44d3-9803-9b33b337ac19","Type":"ContainerStarted","Data":"aa5024dadd33dbd75ecee4b18b3613199e5b9ac0518572483255791fc41fa8b1"}
Aug 04 18:29:33.253252 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:33.253158    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgvv8" event={"ID":"65dfd213-63dd-44d3-9803-9b33b337ac19","Type":"ContainerStarted","Data":"1395421d7234af6cdf367162e77af3dd8c7866f8c91a2b4db11a8c2452080bd6"}
Aug 04 18:29:34.645411 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:34.645385    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:34.746135 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:34.746095    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:34.846392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:34.846314    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:34.937854 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:34.937820    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:35.261942 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:35.261855    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgvv8" event={"ID":"65dfd213-63dd-44d3-9803-9b33b337ac19","Type":"ContainerStarted","Data":"c1ea12d3068b3b3e6a6d1b7659e01006ce6018f0546509ce7651710977b1326c"}
Aug 04 18:29:35.291624 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:35.291442    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zgvv8" podStartSLOduration=17.862306316 podStartE2EDuration="20.291421572s" totalImagesPullingTime="2.429115256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 18:29:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:29:35.290730306 +0000 UTC m=+117.157645407" watchObservedRunningTime="2026-08-04 18:29:35.291421572 +0000 UTC m=+117.158336672"
Aug 04 18:29:35.317685 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:35.317657    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:35.390830 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:35.390802    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:35.849625 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:35.849568    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:35.952973 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:35.952929    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:36.137902 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:36.137809    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:36.350642 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:36.350607    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:36.531804 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:36.531732    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:36.627369 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:36.627339    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:37.055432 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:37.055399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:37.225727 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:37.225694    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:37.552617 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:37.552559    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:37.635722 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:37.635690    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:39.797421 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:39.797390    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:39.878519 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:39.878489    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:40.045449 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:40.045416    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:40.147044 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:40.146966    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:42.849727 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:42.849691    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:42.949477 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:42.949446    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:43.160638 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:43.160531    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:43.385987 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:43.385941    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" podUID="452fb709-0018-44e1-b536-0ed6715b0d65" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 04 18:29:45.936753 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:45.936720    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:46.230179 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.230100    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:46.268323 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.268284    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-45xst"]
Aug 04 18:29:46.270483 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.270461    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.273843 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.273820    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 04 18:29:46.274138 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.274121    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 04 18:29:46.275075 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.275059    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 04 18:29:46.275148 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.275066    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 04 18:29:46.275148 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.275101    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tl7g6\""
Aug 04 18:29:46.275241 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.275107    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 04 18:29:46.275839 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.275820    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 04 18:29:46.351716 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.351681    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:46.369969 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.369943    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9006431f-fe11-4a12-8b69-988b25b48dd8-metrics-client-ca\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.370103 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.369978    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-textfile\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.370103 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.370019    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9006431f-fe11-4a12-8b69-988b25b48dd8-sys\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.370103 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.370037    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42r4n\" (UniqueName: \"kubernetes.io/projected/9006431f-fe11-4a12-8b69-988b25b48dd8-kube-api-access-42r4n\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.370103 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.370066    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.370261 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.370107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9006431f-fe11-4a12-8b69-988b25b48dd8-root\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.370261 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.370131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-tls\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.370261 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.370171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-wtmp\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.370261 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.370191    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-accelerators-collector-config\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.471426 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9006431f-fe11-4a12-8b69-988b25b48dd8-metrics-client-ca\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.471426 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471429    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-textfile\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.471709 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9006431f-fe11-4a12-8b69-988b25b48dd8-sys\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.471709 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42r4n\" (UniqueName: \"kubernetes.io/projected/9006431f-fe11-4a12-8b69-988b25b48dd8-kube-api-access-42r4n\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.471709 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.471709 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9006431f-fe11-4a12-8b69-988b25b48dd8-root\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.471709 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471600    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9006431f-fe11-4a12-8b69-988b25b48dd8-sys\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.471709 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-tls\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.471709 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471697    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9006431f-fe11-4a12-8b69-988b25b48dd8-root\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.472029 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471721    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-wtmp\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.472029 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471755    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-accelerators-collector-config\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.472029 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471893    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-textfile\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.472029 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.471895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-wtmp\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.472195 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.472092    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9006431f-fe11-4a12-8b69-988b25b48dd8-metrics-client-ca\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.472277 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.472259    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-accelerators-collector-config\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.473968 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.473947    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.474078 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.474023    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9006431f-fe11-4a12-8b69-988b25b48dd8-node-exporter-tls\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.480274 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.480225    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42r4n\" (UniqueName: \"kubernetes.io/projected/9006431f-fe11-4a12-8b69-988b25b48dd8-kube-api-access-42r4n\") pod \"node-exporter-45xst\" (UID: \"9006431f-fe11-4a12-8b69-988b25b48dd8\") " pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.489696 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.489675    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:46.578700 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.578663    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-45xst"
Aug 04 18:29:46.587859 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:29:46.587832    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9006431f_fe11_4a12_8b69_988b25b48dd8.slice/crio-6a693e3bd0953e86592f94114e6503b2b05381da99246a61631b8e62a36227a0 WatchSource:0}: Error finding container 6a693e3bd0953e86592f94114e6503b2b05381da99246a61631b8e62a36227a0: Status 404 returned error can't find the container with id 6a693e3bd0953e86592f94114e6503b2b05381da99246a61631b8e62a36227a0
Aug 04 18:29:46.590860 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.590833    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:46.697116 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.697084    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:46.790209 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:46.790177    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:47.208250 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:47.208124    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:47.298134 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:47.298103    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-45xst" event={"ID":"9006431f-fe11-4a12-8b69-988b25b48dd8","Type":"ContainerStarted","Data":"6a693e3bd0953e86592f94114e6503b2b05381da99246a61631b8e62a36227a0"}
Aug 04 18:29:47.310044 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:47.310012    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:47.480873 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:47.480801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:29:47.482979 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:47.482952    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ebbf61e-7a2b-472f-9232-8d9c81f96ded-metrics-certs\") pod \"network-metrics-daemon-fcf2z\" (UID: \"6ebbf61e-7a2b-472f-9232-8d9c81f96ded\") " pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:29:47.521546 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:47.521519    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zw2lq\""
Aug 04 18:29:47.529890 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:47.529862    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fcf2z"
Aug 04 18:29:47.675062 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:47.675029    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:47.795030 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:47.795006    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fcf2z"]
Aug 04 18:29:47.797528 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:29:47.797503    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ebbf61e_7a2b_472f_9232_8d9c81f96ded.slice/crio-9b7f524f716a8aef87fad1c7d56d28ab5ed4498bfddde3db9678ba4b27635152 WatchSource:0}: Error finding container 9b7f524f716a8aef87fad1c7d56d28ab5ed4498bfddde3db9678ba4b27635152: Status 404 returned error can't find the container with id 9b7f524f716a8aef87fad1c7d56d28ab5ed4498bfddde3db9678ba4b27635152
Aug 04 18:29:48.302729 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:48.302695    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fcf2z" event={"ID":"6ebbf61e-7a2b-472f-9232-8d9c81f96ded","Type":"ContainerStarted","Data":"9b7f524f716a8aef87fad1c7d56d28ab5ed4498bfddde3db9678ba4b27635152"}
Aug 04 18:29:48.304143 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:48.304119    2578 generic.go:356] "Generic (PLEG): container finished" podID="9006431f-fe11-4a12-8b69-988b25b48dd8" containerID="63445f8f73fc4cf24ec9174ad958fc981a65dc349e7b3292b0d74d49b1cf8dfb" exitCode=0
Aug 04 18:29:48.304304 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:48.304172    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-45xst" event={"ID":"9006431f-fe11-4a12-8b69-988b25b48dd8","Type":"ContainerDied","Data":"63445f8f73fc4cf24ec9174ad958fc981a65dc349e7b3292b0d74d49b1cf8dfb"}
Aug 04 18:29:48.951657 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:48.951624    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:49.208964 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:49.208886    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:49.296279 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:49.296247    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:49.311358 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:49.311330    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-45xst" event={"ID":"9006431f-fe11-4a12-8b69-988b25b48dd8","Type":"ContainerStarted","Data":"1657ff7bde8594ede794b714fa59ab9f96c973a5400ea0531e35b3093c91c86d"}
Aug 04 18:29:49.311733 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:49.311377    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-45xst" event={"ID":"9006431f-fe11-4a12-8b69-988b25b48dd8","Type":"ContainerStarted","Data":"ce5a2a1d06db17d84ea1c8dd12799929518f56a3907ff88481381a21a1618a53"}
Aug 04 18:29:49.330844 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:49.330802    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-45xst" podStartSLOduration=2.133332461 podStartE2EDuration="3.330786532s" totalImagesPullingTime="1.13201909s" totalInitContainerRuntime="65.434981ms" isStatefulPod=true podCreationTimestamp="2026-08-04 18:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:29:49.329676864 +0000 UTC m=+131.196591965" watchObservedRunningTime="2026-08-04 18:29:49.330786532 +0000 UTC m=+131.197701647"
Aug 04 18:29:49.370305 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:49.370272    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:49.493052 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:49.492966    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:49.952205 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:49.952119    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:50.049679 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:50.049643    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:50.316177 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:50.316146    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fcf2z" event={"ID":"6ebbf61e-7a2b-472f-9232-8d9c81f96ded","Type":"ContainerStarted","Data":"fed4ca35e932609b02f4e3401fe04ff387a23a1e088c621055c2330747d47f40"}
Aug 04 18:29:50.316554 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:50.316204    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fcf2z" event={"ID":"6ebbf61e-7a2b-472f-9232-8d9c81f96ded","Type":"ContainerStarted","Data":"2a5a785cc85807b6454b7d2f58cc10d306e4eb054ed03fc63f2f0b2185ac8724"}
Aug 04 18:29:50.339530 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:50.339479    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fcf2z" podStartSLOduration=130.260061931 podStartE2EDuration="2m12.339462006s" totalImagesPullingTime="2.079400075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:29:50.338668932 +0000 UTC m=+132.205584032" watchObservedRunningTime="2026-08-04 18:29:50.339462006 +0000 UTC m=+132.206377161"
Aug 04 18:29:50.441015 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:50.440978    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:50.588000 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:50.587911    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:50.987945 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:50.987862    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:51.083000 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:51.082961    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:51.197235 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:51.197202    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:52.493922 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:52.493888    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:53.386257 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:53.386221    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" podUID="452fb709-0018-44e1-b536-0ed6715b0d65" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 04 18:29:56.616327 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:56.616287    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:56.751177 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:56.751133    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:57.855338 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:57.855302    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:57.938507 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:57.938476    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:58.264109 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:58.264024    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:58.331589 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:58.331522    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:59.401846 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:59.401812    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-575496654b-z8wsj"]
Aug 04 18:29:59.402237 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:29:59.402074    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-575496654b-z8wsj" podUID="aff1fa38-3505-4972-a71e-2a37e8c3fa23"
Aug 04 18:29:59.503172 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:59.503139    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:29:59.563069 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:29:59.563036    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:00.345488 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.345461    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:30:00.351508 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.351490    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:30:00.485758 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.485713    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aff1fa38-3505-4972-a71e-2a37e8c3fa23-ca-trust-extracted\") pod \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") "
Aug 04 18:30:00.485758 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.485755    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-certificates\") pod \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") "
Aug 04 18:30:00.486300 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.485779    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-installation-pull-secrets\") pod \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") "
Aug 04 18:30:00.486300 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.485827    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-trusted-ca\") pod \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") "
Aug 04 18:30:00.486300 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.485851    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f45pw\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-kube-api-access-f45pw\") pod \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") "
Aug 04 18:30:00.486300 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.485867    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-bound-sa-token\") pod \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") "
Aug 04 18:30:00.486300 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.485890    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-image-registry-private-configuration\") pod \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\" (UID: \"aff1fa38-3505-4972-a71e-2a37e8c3fa23\") "
Aug 04 18:30:00.486300 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.486025    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff1fa38-3505-4972-a71e-2a37e8c3fa23-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "aff1fa38-3505-4972-a71e-2a37e8c3fa23" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:30:00.486300 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.486213    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "aff1fa38-3505-4972-a71e-2a37e8c3fa23" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:30:00.486619 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.486381    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "aff1fa38-3505-4972-a71e-2a37e8c3fa23" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:30:00.488267 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.488233    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "aff1fa38-3505-4972-a71e-2a37e8c3fa23" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:30:00.488267 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.488247    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "aff1fa38-3505-4972-a71e-2a37e8c3fa23" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:30:00.488467 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.488443    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "aff1fa38-3505-4972-a71e-2a37e8c3fa23" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:30:00.488517 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.488465    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-kube-api-access-f45pw" (OuterVolumeSpecName: "kube-api-access-f45pw") pod "aff1fa38-3505-4972-a71e-2a37e8c3fa23" (UID: "aff1fa38-3505-4972-a71e-2a37e8c3fa23"). InnerVolumeSpecName "kube-api-access-f45pw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:30:00.489945 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.489921    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:00.557230 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.557182    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:00.586874 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.586840    2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-image-registry-private-configuration\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:30:00.586874 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.586871    2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aff1fa38-3505-4972-a71e-2a37e8c3fa23-ca-trust-extracted\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:30:00.587053 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.586885    2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-certificates\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:30:00.587053 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.586897    2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aff1fa38-3505-4972-a71e-2a37e8c3fa23-installation-pull-secrets\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:30:00.587053 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.586910    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aff1fa38-3505-4972-a71e-2a37e8c3fa23-trusted-ca\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:30:00.587053 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.586920    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f45pw\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-kube-api-access-f45pw\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:30:00.587053 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:00.586932    2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-bound-sa-token\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:30:01.348869 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:01.348833    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-575496654b-z8wsj"
Aug 04 18:30:01.356177 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:01.356148    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:01.382915 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:01.382882    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-575496654b-z8wsj"]
Aug 04 18:30:01.387458 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:01.387432    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-575496654b-z8wsj"]
Aug 04 18:30:01.493858 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:01.493825    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aff1fa38-3505-4972-a71e-2a37e8c3fa23-registry-tls\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:30:01.512988 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:01.512952    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:02.800357 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:02.800326    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aff1fa38-3505-4972-a71e-2a37e8c3fa23" path="/var/lib/kubelet/pods/aff1fa38-3505-4972-a71e-2a37e8c3fa23/volumes"
Aug 04 18:30:03.385838 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:03.385795    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" podUID="452fb709-0018-44e1-b536-0ed6715b0d65" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 04 18:30:03.385981 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:03.385875    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc"
Aug 04 18:30:03.386443 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:03.386412    2578 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"703fe970b87035ff47fc8a7977d7a268aac12b84b143b3b55dee00211e1ee667"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Aug 04 18:30:03.386484 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:03.386469    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" podUID="452fb709-0018-44e1-b536-0ed6715b0d65" containerName="service-proxy" containerID="cri-o://703fe970b87035ff47fc8a7977d7a268aac12b84b143b3b55dee00211e1ee667" gracePeriod=30
Aug 04 18:30:04.359686 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:04.359655    2578 generic.go:356] "Generic (PLEG): container finished" podID="452fb709-0018-44e1-b536-0ed6715b0d65" containerID="703fe970b87035ff47fc8a7977d7a268aac12b84b143b3b55dee00211e1ee667" exitCode=2
Aug 04 18:30:04.360059 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:04.359739    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" event={"ID":"452fb709-0018-44e1-b536-0ed6715b0d65","Type":"ContainerDied","Data":"703fe970b87035ff47fc8a7977d7a268aac12b84b143b3b55dee00211e1ee667"}
Aug 04 18:30:04.360059 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:04.359785    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695cf4fb46-b4lhc" event={"ID":"452fb709-0018-44e1-b536-0ed6715b0d65","Type":"ContainerStarted","Data":"df933551bff640e59ff5b87cff7b1b94f07fe07d8f97b7018e2d577c2027cfa1"}
Aug 04 18:30:05.159334 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:05.159302    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:05.699145 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:05.699115    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:07.990006 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:07.989975    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:08.096766 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:08.096739    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:09.562478 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:09.562443    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:09.782424 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:09.782390    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:11.645282 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:11.645250    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:11.716190 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:11.716162    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:12.196296 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:12.196265    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:12.299876 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:12.299843    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:15.940087 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:15.940059    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:16.100439 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:30:16.100389    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jwfsp" podUID="1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11"
Aug 04 18:30:16.129768 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:16.129732    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:16.326595 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:16.326554    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:16.397015 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:16.396988    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:30:16.891039 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:16.891004    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:17.065277 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:17.065239    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:21.038629 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:21.038558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:30:21.040867 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:21.040845    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11-metrics-tls\") pod \"dns-default-jwfsp\" (UID: \"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11\") " pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:30:21.201006 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:21.200971    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t8fxw\""
Aug 04 18:30:21.209192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:21.209166    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:30:21.341558 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:21.341412    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jwfsp"]
Aug 04 18:30:21.344157 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:30:21.344128    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4b4d2c_fbc2_4d91_aebd_a0ecf05dfe11.slice/crio-3b4d2abf0fa351c8a462a41cdd88d70762bd97360f8449fb70e38f6f8005a3cb WatchSource:0}: Error finding container 3b4d2abf0fa351c8a462a41cdd88d70762bd97360f8449fb70e38f6f8005a3cb: Status 404 returned error can't find the container with id 3b4d2abf0fa351c8a462a41cdd88d70762bd97360f8449fb70e38f6f8005a3cb
Aug 04 18:30:21.414305 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:21.414271    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jwfsp" event={"ID":"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11","Type":"ContainerStarted","Data":"3b4d2abf0fa351c8a462a41cdd88d70762bd97360f8449fb70e38f6f8005a3cb"}
Aug 04 18:30:21.471491 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:21.471455    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:21.572213 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:21.572121    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:22.390361 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:22.390322    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:22.472630 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:22.472599    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:23.423957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:23.423922    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jwfsp" event={"ID":"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11","Type":"ContainerStarted","Data":"7c1028721ed592b636675d9481f3ebeefecb4efc1534e4d35a7a8a708eaa808e"}
Aug 04 18:30:23.424383 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:23.423983    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jwfsp" event={"ID":"1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11","Type":"ContainerStarted","Data":"bedb450a72d6704564dbee55558654d12a1a5c21f03c4f9be118c4be14598069"}
Aug 04 18:30:23.424383 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:23.424012    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:30:23.446885 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:23.446821    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jwfsp" podStartSLOduration=128.795336499 podStartE2EDuration="2m10.446800562s" totalImagesPullingTime="1.651464063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:28:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:30:23.445869108 +0000 UTC m=+165.312784220" watchObservedRunningTime="2026-08-04 18:30:23.446800562 +0000 UTC m=+165.313715659"
Aug 04 18:30:27.894567 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:27.894533    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:27.992785 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:27.992744    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:33.429760 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:33.429688    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jwfsp"
Aug 04 18:30:42.597939 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:42.597899    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:43.136179 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:43.136145    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:30:43.217235 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:30:43.217197    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:02.369560 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:02.369522    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:02.429230 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:02.429186    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:03.103221 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:03.103190    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:03.204347 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:03.204314    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:13.654031 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:13.653995    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:13.755060 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:13.755017    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:13.829461 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:13.829427    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:14.370311 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:14.370273    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:14.433481 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:14.433454    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:27.911241 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:27.911205    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:28.071278 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:28.071247    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:56.261050 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:56.261008    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:31:56.326968 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:31:56.326933    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:32:38.555137 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:32:38.555104    2578 kubelet.go:1648] "Image garbage collection succeeded"
Aug 04 18:32:39.286229 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:32:39.286195    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:32:39.384676 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:32:39.384644    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:06.419143 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:06.419085    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:07.707872 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:07.707828    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:10.229746 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:10.229707    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:10.333064 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:10.333027    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:12.210782 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:12.210742    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:12.373790 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:12.373756    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:12.489387 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:12.489297    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:12.923154 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:12.923119    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:14.592195 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:14.592144    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:14.792049 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:14.792015    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:17.252028 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:17.251976    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:17.352017 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:17.351982    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:22.392801 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:22.392765    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:23.706247 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:23.706214    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:23.774228 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:23.774188    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:23.932193 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:23.932153    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5rccf"]
Aug 04 18:33:23.935122 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:23.935106    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5rccf"
Aug 04 18:33:23.937739 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:23.937718    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-lssgc\""
Aug 04 18:33:23.937868 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:23.937720    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Aug 04 18:33:23.938770 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:23.938752    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Aug 04 18:33:23.946856 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:23.946836    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5rccf"]
Aug 04 18:33:24.004358 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:24.004255    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7348a1d-8382-437c-acb7-300e0d87d65a-bound-sa-token\") pod \"cert-manager-55b4d954d5-5rccf\" (UID: \"f7348a1d-8382-437c-acb7-300e0d87d65a\") " pod="cert-manager/cert-manager-55b4d954d5-5rccf"
Aug 04 18:33:24.004358 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:24.004323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7fgt\" (UniqueName: \"kubernetes.io/projected/f7348a1d-8382-437c-acb7-300e0d87d65a-kube-api-access-c7fgt\") pod \"cert-manager-55b4d954d5-5rccf\" (UID: \"f7348a1d-8382-437c-acb7-300e0d87d65a\") " pod="cert-manager/cert-manager-55b4d954d5-5rccf"
Aug 04 18:33:24.104697 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:24.104657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7fgt\" (UniqueName: \"kubernetes.io/projected/f7348a1d-8382-437c-acb7-300e0d87d65a-kube-api-access-c7fgt\") pod \"cert-manager-55b4d954d5-5rccf\" (UID: \"f7348a1d-8382-437c-acb7-300e0d87d65a\") " pod="cert-manager/cert-manager-55b4d954d5-5rccf"
Aug 04 18:33:24.104872 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:24.104735    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7348a1d-8382-437c-acb7-300e0d87d65a-bound-sa-token\") pod \"cert-manager-55b4d954d5-5rccf\" (UID: \"f7348a1d-8382-437c-acb7-300e0d87d65a\") " pod="cert-manager/cert-manager-55b4d954d5-5rccf"
Aug 04 18:33:24.113373 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:24.113341    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7348a1d-8382-437c-acb7-300e0d87d65a-bound-sa-token\") pod \"cert-manager-55b4d954d5-5rccf\" (UID: \"f7348a1d-8382-437c-acb7-300e0d87d65a\") " pod="cert-manager/cert-manager-55b4d954d5-5rccf"
Aug 04 18:33:24.113557 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:24.113537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7fgt\" (UniqueName: \"kubernetes.io/projected/f7348a1d-8382-437c-acb7-300e0d87d65a-kube-api-access-c7fgt\") pod \"cert-manager-55b4d954d5-5rccf\" (UID: \"f7348a1d-8382-437c-acb7-300e0d87d65a\") " pod="cert-manager/cert-manager-55b4d954d5-5rccf"
Aug 04 18:33:24.243815 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:24.243771    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5rccf"
Aug 04 18:33:24.368251 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:24.368223    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5rccf"]
Aug 04 18:33:24.371289 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:33:24.371261    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7348a1d_8382_437c_acb7_300e0d87d65a.slice/crio-4bd0e7d2477e4184e7f5e5696f9991ca29d6d2b97a0763d60e8742dcc22d6a6e WatchSource:0}: Error finding container 4bd0e7d2477e4184e7f5e5696f9991ca29d6d2b97a0763d60e8742dcc22d6a6e: Status 404 returned error can't find the container with id 4bd0e7d2477e4184e7f5e5696f9991ca29d6d2b97a0763d60e8742dcc22d6a6e
Aug 04 18:33:24.373229 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:24.373216    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 18:33:24.992533 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:24.992493    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5rccf" event={"ID":"f7348a1d-8382-437c-acb7-300e0d87d65a","Type":"ContainerStarted","Data":"4bd0e7d2477e4184e7f5e5696f9991ca29d6d2b97a0763d60e8742dcc22d6a6e"}
Aug 04 18:33:25.666904 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.666805    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9"]
Aug 04 18:33:25.670482 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.670362    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9"
Aug 04 18:33:25.673010 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.672979    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Aug 04 18:33:25.674122 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.674043    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-mwxff\""
Aug 04 18:33:25.674122 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.674079    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Aug 04 18:33:25.678972 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.678923    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9"]
Aug 04 18:33:25.819598 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.819529    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgrz4\" (UniqueName: \"kubernetes.io/projected/3b57ce7d-5c95-4647-a1d9-e08cee19eaeb-kube-api-access-hgrz4\") pod \"openshift-lws-operator-fd8ccff4c-nvfh9\" (UID: \"3b57ce7d-5c95-4647-a1d9-e08cee19eaeb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9"
Aug 04 18:33:25.819782 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.819624    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b57ce7d-5c95-4647-a1d9-e08cee19eaeb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nvfh9\" (UID: \"3b57ce7d-5c95-4647-a1d9-e08cee19eaeb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9"
Aug 04 18:33:25.921248 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.920929    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b57ce7d-5c95-4647-a1d9-e08cee19eaeb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nvfh9\" (UID: \"3b57ce7d-5c95-4647-a1d9-e08cee19eaeb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9"
Aug 04 18:33:25.921248 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.921036    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgrz4\" (UniqueName: \"kubernetes.io/projected/3b57ce7d-5c95-4647-a1d9-e08cee19eaeb-kube-api-access-hgrz4\") pod \"openshift-lws-operator-fd8ccff4c-nvfh9\" (UID: \"3b57ce7d-5c95-4647-a1d9-e08cee19eaeb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9"
Aug 04 18:33:25.921482 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.921404    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b57ce7d-5c95-4647-a1d9-e08cee19eaeb-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nvfh9\" (UID: \"3b57ce7d-5c95-4647-a1d9-e08cee19eaeb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9"
Aug 04 18:33:25.943607 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.943503    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgrz4\" (UniqueName: \"kubernetes.io/projected/3b57ce7d-5c95-4647-a1d9-e08cee19eaeb-kube-api-access-hgrz4\") pod \"openshift-lws-operator-fd8ccff4c-nvfh9\" (UID: \"3b57ce7d-5c95-4647-a1d9-e08cee19eaeb\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9"
Aug 04 18:33:25.983471 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:25.983424    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9"
Aug 04 18:33:27.423966 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:27.423925    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9"]
Aug 04 18:33:27.430345 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:33:27.430317    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b57ce7d_5c95_4647_a1d9_e08cee19eaeb.slice/crio-2892e0c38d67bb3dcd3e2529b56872e5a6219f0ca42e2b87108c7b973f222d26 WatchSource:0}: Error finding container 2892e0c38d67bb3dcd3e2529b56872e5a6219f0ca42e2b87108c7b973f222d26: Status 404 returned error can't find the container with id 2892e0c38d67bb3dcd3e2529b56872e5a6219f0ca42e2b87108c7b973f222d26
Aug 04 18:33:28.005034 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:28.005001    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5rccf" event={"ID":"f7348a1d-8382-437c-acb7-300e0d87d65a","Type":"ContainerStarted","Data":"2951a8d8a790b380e20b7b8ef33298fccdde988ce67d1c10c4e03a6076ed1213"}
Aug 04 18:33:28.006244 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:28.006218    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9" event={"ID":"3b57ce7d-5c95-4647-a1d9-e08cee19eaeb","Type":"ContainerStarted","Data":"2892e0c38d67bb3dcd3e2529b56872e5a6219f0ca42e2b87108c7b973f222d26"}
Aug 04 18:33:28.037416 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:28.037353    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-5rccf" podStartSLOduration=2.056393306 podStartE2EDuration="5.037334493s" totalImagesPullingTime="2.980941187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:33:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:33:28.036342992 +0000 UTC m=+349.903258097" watchObservedRunningTime="2026-08-04 18:33:28.037334493 +0000 UTC m=+349.904249594"
Aug 04 18:33:30.014785 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:30.014743    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9" event={"ID":"3b57ce7d-5c95-4647-a1d9-e08cee19eaeb","Type":"ContainerStarted","Data":"8364b42b8e3affeee922f0d6c6520a86f23f227fe8a7259e831be8b23fdf9e1a"}
Aug 04 18:33:30.034691 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:30.034639    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nvfh9" podStartSLOduration=2.584559589 podStartE2EDuration="5.03462293s" totalImagesPullingTime="2.450063341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:33:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:33:30.033334688 +0000 UTC m=+351.900249790" watchObservedRunningTime="2026-08-04 18:33:30.03462293 +0000 UTC m=+351.901538030"
Aug 04 18:33:30.638786 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:30.638751    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:30.739857 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:30.739785    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:32.903277 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:32.903241    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:45.226747 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:45.226699    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:46.487760 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.487727    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:46.594735 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.594702    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"]
Aug 04 18:33:46.600806 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.600787    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.603875 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.603851    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 04 18:33:46.604013 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.603968    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-v6z2v\""
Aug 04 18:33:46.604013 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.604004    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 04 18:33:46.604116 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.604054    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 04 18:33:46.604162 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.604133    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 04 18:33:46.631345 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.631318    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"]
Aug 04 18:33:46.680873 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.680836    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmhkt\" (UniqueName: \"kubernetes.io/projected/1d8e147b-4158-41ec-87da-f1c21ce06121-kube-api-access-bmhkt\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.681033 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.680885    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d8e147b-4158-41ec-87da-f1c21ce06121-webhook-cert\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.681033 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.680936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d8e147b-4158-41ec-87da-f1c21ce06121-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.681033 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.680975    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d8e147b-4158-41ec-87da-f1c21ce06121-tmp\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.681033 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.680992    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d8e147b-4158-41ec-87da-f1c21ce06121-manifests\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.782358 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.782282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d8e147b-4158-41ec-87da-f1c21ce06121-webhook-cert\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.782358 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.782324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d8e147b-4158-41ec-87da-f1c21ce06121-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.782358 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.782344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d8e147b-4158-41ec-87da-f1c21ce06121-tmp\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.782620 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.782484    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d8e147b-4158-41ec-87da-f1c21ce06121-manifests\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.782686 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.782641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmhkt\" (UniqueName: \"kubernetes.io/projected/1d8e147b-4158-41ec-87da-f1c21ce06121-kube-api-access-bmhkt\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.782903 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.782874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d8e147b-4158-41ec-87da-f1c21ce06121-manifests\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.784728 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.784693    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d8e147b-4158-41ec-87da-f1c21ce06121-tmp\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.784844 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.784830    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d8e147b-4158-41ec-87da-f1c21ce06121-webhook-cert\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.784929 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.784911    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d8e147b-4158-41ec-87da-f1c21ce06121-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.789458 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.789434    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:46.794287 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.794266    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmhkt\" (UniqueName: \"kubernetes.io/projected/1d8e147b-4158-41ec-87da-f1c21ce06121-kube-api-access-bmhkt\") pod \"opendatahub-operator-controller-manager-5556d6485b-cs9tl\" (UID: \"1d8e147b-4158-41ec-87da-f1c21ce06121\") " pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:46.891684 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.891646    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:46.910829 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:46.910793    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:47.072316 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:47.072253    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:47.127955 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:33:47.127925    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d8e147b_4158_41ec_87da_f1c21ce06121.slice/crio-0d555dc74e214839113211de59e2e435fb8ce0e4382dd6abab5d0c95a10f42b9 WatchSource:0}: Error finding container 0d555dc74e214839113211de59e2e435fb8ce0e4382dd6abab5d0c95a10f42b9: Status 404 returned error can't find the container with id 0d555dc74e214839113211de59e2e435fb8ce0e4382dd6abab5d0c95a10f42b9
Aug 04 18:33:47.130781 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:47.130761    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"]
Aug 04 18:33:47.283044 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:47.283011    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:47.404331 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:47.404253    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:47.502850 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:47.502812    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:47.605908 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:47.605864    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:48.079342 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:48.079299    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl" event={"ID":"1d8e147b-4158-41ec-87da-f1c21ce06121","Type":"ContainerStarted","Data":"0d555dc74e214839113211de59e2e435fb8ce0e4382dd6abab5d0c95a10f42b9"}
Aug 04 18:33:49.056499 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:49.056463    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:49.184773 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:49.184729    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:50.088349 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:50.088312    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl" event={"ID":"1d8e147b-4158-41ec-87da-f1c21ce06121","Type":"ContainerStarted","Data":"eee1c86710ec667e140acf26529d4eeb575a502a34fa5cf8088b0cf1869eac9d"}
Aug 04 18:33:52.095925 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:52.095890    2578 generic.go:356] "Generic (PLEG): container finished" podID="1d8e147b-4158-41ec-87da-f1c21ce06121" containerID="eee1c86710ec667e140acf26529d4eeb575a502a34fa5cf8088b0cf1869eac9d" exitCode=0
Aug 04 18:33:52.096330 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:52.095941    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl" event={"ID":"1d8e147b-4158-41ec-87da-f1c21ce06121","Type":"ContainerDied","Data":"eee1c86710ec667e140acf26529d4eeb575a502a34fa5cf8088b0cf1869eac9d"}
Aug 04 18:33:53.101569 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:53.101479    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl" event={"ID":"1d8e147b-4158-41ec-87da-f1c21ce06121","Type":"ContainerStarted","Data":"961833a2c85e444b3db8922bfa0921afb3c9e3a72e6990e850444798954bb5ce"}
Aug 04 18:33:53.101931 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:53.101726    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:33:53.144321 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:53.144263    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl" podStartSLOduration=2.510776426 podStartE2EDuration="7.144246973s" totalImagesPullingTime="3.473250901s" totalInitContainerRuntime="1.160219646s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:33:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:33:53.141823081 +0000 UTC m=+375.008738172" watchObservedRunningTime="2026-08-04 18:33:53.144246973 +0000 UTC m=+375.011162073"
Aug 04 18:33:57.626776 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:57.626738    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:33:57.803827 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:33:57.803790    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:01.403845 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:01.403810    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:01.784775 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:01.784687    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:04.107513 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:04.107480    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5556d6485b-cs9tl"
Aug 04 18:34:06.637286 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:06.637251    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:06.969711 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:06.969625    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:07.359244 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.359207    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"]
Aug 04 18:34:07.362211 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.362188    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:07.365878 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.365853    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cvgpz\""
Aug 04 18:34:07.366038 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.365898    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 04 18:34:07.366038 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.365896    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 04 18:34:07.366206 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.366191    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 04 18:34:07.366924 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.366908    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 04 18:34:07.383666 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.383636    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:07.387779 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.387753    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"]
Aug 04 18:34:07.440637 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.440571    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw2qz\" (UniqueName: \"kubernetes.io/projected/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-kube-api-access-sw2qz\") pod \"kube-auth-proxy-65f654896c-bgwnz\" (UID: \"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2\") " pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:07.440825 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.440662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-tls-certs\") pod \"kube-auth-proxy-65f654896c-bgwnz\" (UID: \"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2\") " pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:07.440825 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.440682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-tmp\") pod \"kube-auth-proxy-65f654896c-bgwnz\" (UID: \"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2\") " pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:07.541699 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.541657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-tls-certs\") pod \"kube-auth-proxy-65f654896c-bgwnz\" (UID: \"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2\") " pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:07.541699 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.541697    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-tmp\") pod \"kube-auth-proxy-65f654896c-bgwnz\" (UID: \"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2\") " pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:07.541908 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.541743    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw2qz\" (UniqueName: \"kubernetes.io/projected/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-kube-api-access-sw2qz\") pod \"kube-auth-proxy-65f654896c-bgwnz\" (UID: \"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2\") " pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:07.541908 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:34:07.541816    2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Aug 04 18:34:07.541908 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:34:07.541896    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-tls-certs podName:7ac5c5b2-4c34-4398-aa0e-d1907eae44e2 nodeName:}" failed. No retries permitted until 2026-08-04 18:34:08.041876651 +0000 UTC m=+389.908791730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-tls-certs") pod "kube-auth-proxy-65f654896c-bgwnz" (UID: "7ac5c5b2-4c34-4398-aa0e-d1907eae44e2") : secret "kube-auth-proxy-tls" not found
Aug 04 18:34:07.543941 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.543915    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-tmp\") pod \"kube-auth-proxy-65f654896c-bgwnz\" (UID: \"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2\") " pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:07.565238 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.565210    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw2qz\" (UniqueName: \"kubernetes.io/projected/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-kube-api-access-sw2qz\") pod \"kube-auth-proxy-65f654896c-bgwnz\" (UID: \"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2\") " pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:07.784165 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.784086    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-55rcx"]
Aug 04 18:34:07.787087 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.787070    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:07.790175 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.790146    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Aug 04 18:34:07.790320 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.790215    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qb9jh\""
Aug 04 18:34:07.805757 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.805726    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-55rcx"]
Aug 04 18:34:07.831747 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.831712    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:07.844898 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.844868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7a08462-4239-49ba-836c-363b6fa441bb-cert\") pod \"odh-model-controller-7998c67dc4-55rcx\" (UID: \"b7a08462-4239-49ba-836c-363b6fa441bb\") " pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:07.845074 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.844936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87nnk\" (UniqueName: \"kubernetes.io/projected/b7a08462-4239-49ba-836c-363b6fa441bb-kube-api-access-87nnk\") pod \"odh-model-controller-7998c67dc4-55rcx\" (UID: \"b7a08462-4239-49ba-836c-363b6fa441bb\") " pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:07.945613 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.945542    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87nnk\" (UniqueName: \"kubernetes.io/projected/b7a08462-4239-49ba-836c-363b6fa441bb-kube-api-access-87nnk\") pod \"odh-model-controller-7998c67dc4-55rcx\" (UID: \"b7a08462-4239-49ba-836c-363b6fa441bb\") " pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:07.945767 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.945642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7a08462-4239-49ba-836c-363b6fa441bb-cert\") pod \"odh-model-controller-7998c67dc4-55rcx\" (UID: \"b7a08462-4239-49ba-836c-363b6fa441bb\") " pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:07.945767 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:34:07.945724    2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Aug 04 18:34:07.945834 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:34:07.945779    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7a08462-4239-49ba-836c-363b6fa441bb-cert podName:b7a08462-4239-49ba-836c-363b6fa441bb nodeName:}" failed. No retries permitted until 2026-08-04 18:34:08.445762209 +0000 UTC m=+390.312677288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b7a08462-4239-49ba-836c-363b6fa441bb-cert") pod "odh-model-controller-7998c67dc4-55rcx" (UID: "b7a08462-4239-49ba-836c-363b6fa441bb") : secret "odh-model-controller-webhook-cert" not found
Aug 04 18:34:07.987205 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:07.987165    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87nnk\" (UniqueName: \"kubernetes.io/projected/b7a08462-4239-49ba-836c-363b6fa441bb-kube-api-access-87nnk\") pod \"odh-model-controller-7998c67dc4-55rcx\" (UID: \"b7a08462-4239-49ba-836c-363b6fa441bb\") " pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:08.034527 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:08.034440    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:08.046450 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:08.046412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-tls-certs\") pod \"kube-auth-proxy-65f654896c-bgwnz\" (UID: \"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2\") " pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:08.048933 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:08.048906    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ac5c5b2-4c34-4398-aa0e-d1907eae44e2-tls-certs\") pod \"kube-auth-proxy-65f654896c-bgwnz\" (UID: \"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2\") " pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:08.251860 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:08.251823    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:08.271846 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:08.271816    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"
Aug 04 18:34:08.448925 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:08.448870    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7a08462-4239-49ba-836c-363b6fa441bb-cert\") pod \"odh-model-controller-7998c67dc4-55rcx\" (UID: \"b7a08462-4239-49ba-836c-363b6fa441bb\") " pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:08.449098 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:34:08.449023    2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Aug 04 18:34:08.449098 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:34:08.449089    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7a08462-4239-49ba-836c-363b6fa441bb-cert podName:b7a08462-4239-49ba-836c-363b6fa441bb nodeName:}" failed. No retries permitted until 2026-08-04 18:34:09.44907205 +0000 UTC m=+391.315987130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b7a08462-4239-49ba-836c-363b6fa441bb-cert") pod "odh-model-controller-7998c67dc4-55rcx" (UID: "b7a08462-4239-49ba-836c-363b6fa441bb") : secret "odh-model-controller-webhook-cert" not found
Aug 04 18:34:08.458375 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:08.458341    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65f654896c-bgwnz"]
Aug 04 18:34:08.461362 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:34:08.461339    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac5c5b2_4c34_4398_aa0e_d1907eae44e2.slice/crio-4714e5f1ddaf5f9818ac465f63c0e3b93c3182308b256d8495d20021c1114ca0 WatchSource:0}: Error finding container 4714e5f1ddaf5f9818ac465f63c0e3b93c3182308b256d8495d20021c1114ca0: Status 404 returned error can't find the container with id 4714e5f1ddaf5f9818ac465f63c0e3b93c3182308b256d8495d20021c1114ca0
Aug 04 18:34:08.611702 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:08.611616    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:08.783786 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:08.783755    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:09.154617 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.154557    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz" event={"ID":"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2","Type":"ContainerStarted","Data":"4714e5f1ddaf5f9818ac465f63c0e3b93c3182308b256d8495d20021c1114ca0"}
Aug 04 18:34:09.456732 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.456636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7a08462-4239-49ba-836c-363b6fa441bb-cert\") pod \"odh-model-controller-7998c67dc4-55rcx\" (UID: \"b7a08462-4239-49ba-836c-363b6fa441bb\") " pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:09.458672 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.458637    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:09.459522 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.459495    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7a08462-4239-49ba-836c-363b6fa441bb-cert\") pod \"odh-model-controller-7998c67dc4-55rcx\" (UID: \"b7a08462-4239-49ba-836c-363b6fa441bb\") " pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:09.598003 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.597943    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:09.658376 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.658188    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v6srd"]
Aug 04 18:34:09.661932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.661904    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-v6srd"
Aug 04 18:34:09.665531 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.665411    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Aug 04 18:34:09.665531 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.665448    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-9sxmx\""
Aug 04 18:34:09.683161 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.683130    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v6srd"]
Aug 04 18:34:09.758873 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.758777    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a1c5ee-79b3-4ac8-8e98-419be848af7d-tls-certs\") pod \"model-serving-api-54d89bd5fc-v6srd\" (UID: \"23a1c5ee-79b3-4ac8-8e98-419be848af7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-v6srd"
Aug 04 18:34:09.758873 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.758850    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh2fk\" (UniqueName: \"kubernetes.io/projected/23a1c5ee-79b3-4ac8-8e98-419be848af7d-kube-api-access-fh2fk\") pod \"model-serving-api-54d89bd5fc-v6srd\" (UID: \"23a1c5ee-79b3-4ac8-8e98-419be848af7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-v6srd"
Aug 04 18:34:09.772587 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.772481    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-55rcx"]
Aug 04 18:34:09.777041 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:34:09.776731    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a08462_4239_49ba_836c_363b6fa441bb.slice/crio-4b3563139dc05b852004a82c34797fe45212b024ee247df8d085f9c97a64575c WatchSource:0}: Error finding container 4b3563139dc05b852004a82c34797fe45212b024ee247df8d085f9c97a64575c: Status 404 returned error can't find the container with id 4b3563139dc05b852004a82c34797fe45212b024ee247df8d085f9c97a64575c
Aug 04 18:34:09.778195 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.778119    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:09.859896 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.859848    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh2fk\" (UniqueName: \"kubernetes.io/projected/23a1c5ee-79b3-4ac8-8e98-419be848af7d-kube-api-access-fh2fk\") pod \"model-serving-api-54d89bd5fc-v6srd\" (UID: \"23a1c5ee-79b3-4ac8-8e98-419be848af7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-v6srd"
Aug 04 18:34:09.860092 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.859964    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a1c5ee-79b3-4ac8-8e98-419be848af7d-tls-certs\") pod \"model-serving-api-54d89bd5fc-v6srd\" (UID: \"23a1c5ee-79b3-4ac8-8e98-419be848af7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-v6srd"
Aug 04 18:34:09.873112 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.873074    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a1c5ee-79b3-4ac8-8e98-419be848af7d-tls-certs\") pod \"model-serving-api-54d89bd5fc-v6srd\" (UID: \"23a1c5ee-79b3-4ac8-8e98-419be848af7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-v6srd"
Aug 04 18:34:09.876003 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.875969    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh2fk\" (UniqueName: \"kubernetes.io/projected/23a1c5ee-79b3-4ac8-8e98-419be848af7d-kube-api-access-fh2fk\") pod \"model-serving-api-54d89bd5fc-v6srd\" (UID: \"23a1c5ee-79b3-4ac8-8e98-419be848af7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-v6srd"
Aug 04 18:34:09.978377 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.978106    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-v6srd"
Aug 04 18:34:09.979070 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.978999    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:09.993846 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.993802    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"]
Aug 04 18:34:09.997490 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:09.997463    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.000205 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.000178    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Aug 04 18:34:10.000335 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.000179    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Aug 04 18:34:10.001211 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.000874    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6q86t\""
Aug 04 18:34:10.001211 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.000983    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Aug 04 18:34:10.001211 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.001080    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Aug 04 18:34:10.013728 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.013653    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"]
Aug 04 18:34:10.062071 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.062040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.062071 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.062077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.062297 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.062100    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8s86\" (UniqueName: \"kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-kube-api-access-h8s86\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.062297 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.062125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.062297 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.062148    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.062297 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.062176    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.062297 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.062201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.075643 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.075592    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:10.153457 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.153330    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v6srd"]
Aug 04 18:34:10.155906 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:34:10.155869    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a1c5ee_79b3_4ac8_8e98_419be848af7d.slice/crio-764d9570b46e54d41e7c10ab16b17ffd31ecbd77b20df02baa0ab909adc6986c WatchSource:0}: Error finding container 764d9570b46e54d41e7c10ab16b17ffd31ecbd77b20df02baa0ab909adc6986c: Status 404 returned error can't find the container with id 764d9570b46e54d41e7c10ab16b17ffd31ecbd77b20df02baa0ab909adc6986c
Aug 04 18:34:10.160406 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.160375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-55rcx" event={"ID":"b7a08462-4239-49ba-836c-363b6fa441bb","Type":"ContainerStarted","Data":"4b3563139dc05b852004a82c34797fe45212b024ee247df8d085f9c97a64575c"}
Aug 04 18:34:10.162852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.162758    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.162852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.162818    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.162852 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.162845    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.163035 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.162868    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8s86\" (UniqueName: \"kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-kube-api-access-h8s86\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.163035 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.162898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.163394 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.163373    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.163463 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.163424    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.163689 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.163666    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.165876 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.165849    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.165974 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.165894    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.166124 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.166100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.166674 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.166657    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.173162 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.173136    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.173497 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.173479    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8s86\" (UniqueName: \"kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-kube-api-access-h8s86\") pod \"istiod-openshift-gateway-7b7995d5cd-75zgg\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:10.312142 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:10.312105    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:11.142423 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:11.142389    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"]
Aug 04 18:34:11.165861 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:11.165823    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-v6srd" event={"ID":"23a1c5ee-79b3-4ac8-8e98-419be848af7d","Type":"ContainerStarted","Data":"764d9570b46e54d41e7c10ab16b17ffd31ecbd77b20df02baa0ab909adc6986c"}
Aug 04 18:34:11.662475 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:11.662439    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:11.931179 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:34:11.931095    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf373b7ce_5bd6_4622_8e07_bcf6a825ca10.slice/crio-d962384308f5254db530d79264d9981c7852339f8ae47b4661e0732007bfc7d7 WatchSource:0}: Error finding container d962384308f5254db530d79264d9981c7852339f8ae47b4661e0732007bfc7d7: Status 404 returned error can't find the container with id d962384308f5254db530d79264d9981c7852339f8ae47b4661e0732007bfc7d7
Aug 04 18:34:12.171189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:12.171153    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg" event={"ID":"f373b7ce-5bd6-4622-8e07-bcf6a825ca10","Type":"ContainerStarted","Data":"d962384308f5254db530d79264d9981c7852339f8ae47b4661e0732007bfc7d7"}
Aug 04 18:34:13.691119 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:13.691092    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:14.181398 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:14.181361    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-55rcx" event={"ID":"b7a08462-4239-49ba-836c-363b6fa441bb","Type":"ContainerStarted","Data":"5c83bcb11beb033af5e6a07120ac934d7ced75764eaf3de7daf0e8cd32f406bb"}
Aug 04 18:34:14.181616 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:14.181554    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:14.183243 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:14.183213    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-v6srd" event={"ID":"23a1c5ee-79b3-4ac8-8e98-419be848af7d","Type":"ContainerStarted","Data":"cfb69c41d8d2b07d4cfe9ab1f9038780e2a6af7dc3ee3e2d5fa53afd5a7613c2"}
Aug 04 18:34:14.183378 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:14.183319    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-v6srd"
Aug 04 18:34:14.184906 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:14.184882    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz" event={"ID":"7ac5c5b2-4c34-4398-aa0e-d1907eae44e2","Type":"ContainerStarted","Data":"29880fb1b40bc9d2180d016d803ed7b9d14671ae08c3a9812bd07f48404dde21"}
Aug 04 18:34:14.217833 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:14.217745    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-55rcx" podStartSLOduration=3.3692391170000002 podStartE2EDuration="7.217724829s" totalImagesPullingTime="3.848485712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:34:14.217278718 +0000 UTC m=+396.084193819" watchObservedRunningTime="2026-08-04 18:34:14.217724829 +0000 UTC m=+396.084639929"
Aug 04 18:34:14.284919 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:14.284852    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65f654896c-bgwnz" podStartSLOduration=2.1764948410000002 podStartE2EDuration="7.28483035s" totalImagesPullingTime="5.108335509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:34:14.284569719 +0000 UTC m=+396.151484824" watchObservedRunningTime="2026-08-04 18:34:14.28483035 +0000 UTC m=+396.151745452"
Aug 04 18:34:15.400181 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:15.400129    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:34:15.400564 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:15.400220    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:34:16.195875 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:16.195839    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg" event={"ID":"f373b7ce-5bd6-4622-8e07-bcf6a825ca10","Type":"ContainerStarted","Data":"54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349"}
Aug 04 18:34:16.196122 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:16.196026    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:16.197699 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:16.197660    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-75zgg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 18:34:16.197895 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:16.197724    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg" podUID="f373b7ce-5bd6-4622-8e07-bcf6a825ca10" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 18:34:16.272681 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:16.272627    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg" podStartSLOduration=3.807730231 podStartE2EDuration="7.272608565s" totalImagesPullingTime="3.464878334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:34:16.269816333 +0000 UTC m=+398.136731433" watchObservedRunningTime="2026-08-04 18:34:16.272608565 +0000 UTC m=+398.139523666"
Aug 04 18:34:16.272926 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:16.272906    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-v6srd" podStartSLOduration=3.855869936 podStartE2EDuration="7.272901394s" totalImagesPullingTime="3.417031458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:34:14.352041886 +0000 UTC m=+396.218956987" watchObservedRunningTime="2026-08-04 18:34:16.272901394 +0000 UTC m=+398.139816494"
Aug 04 18:34:16.982062 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:16.982009    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:17.200290 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.200260    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-75zgg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 18:34:17.200451 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.200312    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg" podUID="f373b7ce-5bd6-4622-8e07-bcf6a825ca10" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 18:34:17.326021 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.325971    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:17.699960 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.699884    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"]
Aug 04 18:34:17.703423 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.703401    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.706709 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.706688    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-pdcdv\""
Aug 04 18:34:17.729811 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.729776    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"]
Aug 04 18:34:17.732401 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.732375    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.732512 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.732417    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.732512 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.732441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.732512 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.732495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.732677 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.732520    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vvch\" (UniqueName: \"kubernetes.io/projected/44c62d1c-6255-4540-bb8a-ae34288d1b0f-kube-api-access-5vvch\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.732677 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.732550    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.732677 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.732644    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.732778 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.732694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.732778 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.732714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.771363 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.771329    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:17.833326 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.833287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.833514 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.833335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.833514 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.833353    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.833514 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.833380    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.833514 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.833455    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vvch\" (UniqueName: \"kubernetes.io/projected/44c62d1c-6255-4540-bb8a-ae34288d1b0f-kube-api-access-5vvch\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.833753 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.833519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.833753 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.833615    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.833753 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.833646    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.833753 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.833670    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.833945 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.833788    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.833945 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.833850    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.834073 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.834048    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.834139 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.834088    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.834139 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.834109    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.836198 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.836162    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.836339 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.836315    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.858089 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.858047    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vvch\" (UniqueName: \"kubernetes.io/projected/44c62d1c-6255-4540-bb8a-ae34288d1b0f-kube-api-access-5vvch\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:17.858089 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:17.858083    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44c62d1c-6255-4540-bb8a-ae34288d1b0f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht\" (UID: \"44c62d1c-6255-4540-bb8a-ae34288d1b0f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:18.013315 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:18.013225    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:18.132927 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:18.132892    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:18.183596 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:18.183433    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"]
Aug 04 18:34:18.186227 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:34:18.186193    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c62d1c_6255_4540_bb8a_ae34288d1b0f.slice/crio-17c909b80eb247fac7e199679449e3e4c28f8ee2ee5a243fd062c966806f7022 WatchSource:0}: Error finding container 17c909b80eb247fac7e199679449e3e4c28f8ee2ee5a243fd062c966806f7022: Status 404 returned error can't find the container with id 17c909b80eb247fac7e199679449e3e4c28f8ee2ee5a243fd062c966806f7022
Aug 04 18:34:18.204390 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:18.204353    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht" event={"ID":"44c62d1c-6255-4540-bb8a-ae34288d1b0f","Type":"ContainerStarted","Data":"17c909b80eb247fac7e199679449e3e4c28f8ee2ee5a243fd062c966806f7022"}
Aug 04 18:34:18.311885 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:18.311841    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:20.201088 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.201051    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:20.499261 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.499165    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:20.765562 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.765477    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"]
Aug 04 18:34:20.772318 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.772284    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"]
Aug 04 18:34:20.772498 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.772412    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"
Aug 04 18:34:20.775947 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.775917    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"
Aug 04 18:34:20.786817 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.786788    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Aug 04 18:34:20.787026 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.787008    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-hq94n\""
Aug 04 18:34:20.797590 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.797551    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-njrvj\""
Aug 04 18:34:20.797756 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.797622    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Aug 04 18:34:20.819703 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.819661    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"]
Aug 04 18:34:20.864562 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.864522    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f34e173-c23c-401d-ac4d-1606b9139768-cert\") pod \"kserve-controller-manager-d6f4b65b8-nhls4\" (UID: \"1f34e173-c23c-401d-ac4d-1606b9139768\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"
Aug 04 18:34:20.864755 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.864653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frdsl\" (UniqueName: \"kubernetes.io/projected/7c6ca52c-4af7-4bbe-a80b-23c5d1b59064-kube-api-access-frdsl\") pod \"llmisvc-controller-manager-777c6bd7d5-pd99f\" (UID: \"7c6ca52c-4af7-4bbe-a80b-23c5d1b59064\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"
Aug 04 18:34:20.864755 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.864687    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xhvs\" (UniqueName: \"kubernetes.io/projected/1f34e173-c23c-401d-ac4d-1606b9139768-kube-api-access-8xhvs\") pod \"kserve-controller-manager-d6f4b65b8-nhls4\" (UID: \"1f34e173-c23c-401d-ac4d-1606b9139768\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"
Aug 04 18:34:20.864859 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.864751    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c6ca52c-4af7-4bbe-a80b-23c5d1b59064-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pd99f\" (UID: \"7c6ca52c-4af7-4bbe-a80b-23c5d1b59064\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"
Aug 04 18:34:20.878430 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.878399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:20.948418 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.948380    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"]
Aug 04 18:34:20.965726 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.965685    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f34e173-c23c-401d-ac4d-1606b9139768-cert\") pod \"kserve-controller-manager-d6f4b65b8-nhls4\" (UID: \"1f34e173-c23c-401d-ac4d-1606b9139768\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"
Aug 04 18:34:20.965919 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.965767    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frdsl\" (UniqueName: \"kubernetes.io/projected/7c6ca52c-4af7-4bbe-a80b-23c5d1b59064-kube-api-access-frdsl\") pod \"llmisvc-controller-manager-777c6bd7d5-pd99f\" (UID: \"7c6ca52c-4af7-4bbe-a80b-23c5d1b59064\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"
Aug 04 18:34:20.965919 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.965786    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xhvs\" (UniqueName: \"kubernetes.io/projected/1f34e173-c23c-401d-ac4d-1606b9139768-kube-api-access-8xhvs\") pod \"kserve-controller-manager-d6f4b65b8-nhls4\" (UID: \"1f34e173-c23c-401d-ac4d-1606b9139768\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"
Aug 04 18:34:20.965919 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.965804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c6ca52c-4af7-4bbe-a80b-23c5d1b59064-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pd99f\" (UID: \"7c6ca52c-4af7-4bbe-a80b-23c5d1b59064\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"
Aug 04 18:34:20.965919 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:34:20.965856    2578 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Aug 04 18:34:20.966101 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:34:20.965940    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f34e173-c23c-401d-ac4d-1606b9139768-cert podName:1f34e173-c23c-401d-ac4d-1606b9139768 nodeName:}" failed. No retries permitted until 2026-08-04 18:34:21.465917585 +0000 UTC m=+403.332832664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1f34e173-c23c-401d-ac4d-1606b9139768-cert") pod "kserve-controller-manager-d6f4b65b8-nhls4" (UID: "1f34e173-c23c-401d-ac4d-1606b9139768") : secret "kserve-webhook-server-cert" not found
Aug 04 18:34:20.968297 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.968270    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c6ca52c-4af7-4bbe-a80b-23c5d1b59064-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pd99f\" (UID: \"7c6ca52c-4af7-4bbe-a80b-23c5d1b59064\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"
Aug 04 18:34:20.989221 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.989187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frdsl\" (UniqueName: \"kubernetes.io/projected/7c6ca52c-4af7-4bbe-a80b-23c5d1b59064-kube-api-access-frdsl\") pod \"llmisvc-controller-manager-777c6bd7d5-pd99f\" (UID: \"7c6ca52c-4af7-4bbe-a80b-23c5d1b59064\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"
Aug 04 18:34:20.989359 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:20.989187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xhvs\" (UniqueName: \"kubernetes.io/projected/1f34e173-c23c-401d-ac4d-1606b9139768-kube-api-access-8xhvs\") pod \"kserve-controller-manager-d6f4b65b8-nhls4\" (UID: \"1f34e173-c23c-401d-ac4d-1606b9139768\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"
Aug 04 18:34:21.054678 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.054643    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:34:21.054747 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.054715    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:34:21.054789 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.054748    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:34:21.094425 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.094395    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"
Aug 04 18:34:21.150961 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.150900    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:21.220716 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.220681    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht" event={"ID":"44c62d1c-6255-4540-bb8a-ae34288d1b0f","Type":"ContainerStarted","Data":"68b2b66cab79680637dba5179aee3631ec595c6b7c94d44cb471192f6ed269df"}
Aug 04 18:34:21.307182 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:34:21.307151    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7c6ca52c_4af7_4bbe_a80b_23c5d1b59064.slice/crio-0cc5c5428ed869bdb5f4405ccab96b5430b64c1a68f6a34f408bf3dda4d566fc WatchSource:0}: Error finding container 0cc5c5428ed869bdb5f4405ccab96b5430b64c1a68f6a34f408bf3dda4d566fc: Status 404 returned error can't find the container with id 0cc5c5428ed869bdb5f4405ccab96b5430b64c1a68f6a34f408bf3dda4d566fc
Aug 04 18:34:21.336111 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.336061    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht" podStartSLOduration=1.46987926 podStartE2EDuration="4.336043126s" totalImagesPullingTime="2.866163866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:34:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:34:21.303285591 +0000 UTC m=+403.170200705" watchObservedRunningTime="2026-08-04 18:34:21.336043126 +0000 UTC m=+403.202958226"
Aug 04 18:34:21.338323 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.338301    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"]
Aug 04 18:34:21.421370 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.421326    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:21.472156 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.472112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f34e173-c23c-401d-ac4d-1606b9139768-cert\") pod \"kserve-controller-manager-d6f4b65b8-nhls4\" (UID: \"1f34e173-c23c-401d-ac4d-1606b9139768\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"
Aug 04 18:34:21.474629 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.474606    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f34e173-c23c-401d-ac4d-1606b9139768-cert\") pod \"kserve-controller-manager-d6f4b65b8-nhls4\" (UID: \"1f34e173-c23c-401d-ac4d-1606b9139768\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"
Aug 04 18:34:21.688153 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.688056    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"
Aug 04 18:34:21.939444 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:21.939416    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"]
Aug 04 18:34:21.940958 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:34:21.940930    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f34e173_c23c_401d_ac4d_1606b9139768.slice/crio-582077e13f7191204e688a814ef5fa7e7febc83d5ba215576000121504792800 WatchSource:0}: Error finding container 582077e13f7191204e688a814ef5fa7e7febc83d5ba215576000121504792800: Status 404 returned error can't find the container with id 582077e13f7191204e688a814ef5fa7e7febc83d5ba215576000121504792800
Aug 04 18:34:22.013365 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:22.013329    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:22.018318 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:22.018291    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:22.227701 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:22.227616    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f" event={"ID":"7c6ca52c-4af7-4bbe-a80b-23c5d1b59064","Type":"ContainerStarted","Data":"0cc5c5428ed869bdb5f4405ccab96b5430b64c1a68f6a34f408bf3dda4d566fc"}
Aug 04 18:34:22.228965 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:22.228931    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4" event={"ID":"1f34e173-c23c-401d-ac4d-1606b9139768","Type":"ContainerStarted","Data":"582077e13f7191204e688a814ef5fa7e7febc83d5ba215576000121504792800"}
Aug 04 18:34:22.229248 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:22.229229    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:22.230154 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:22.230134    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht"
Aug 04 18:34:22.291671 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:22.291635    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:22.575803 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:22.575760    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:23.583719 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.583675    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"]
Aug 04 18:34:23.584206 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.583948    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg" podUID="f373b7ce-5bd6-4622-8e07-bcf6a825ca10" containerName="discovery" containerID="cri-o://54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349" gracePeriod=30
Aug 04 18:34:23.845254 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.845186    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:23.894910 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.894880    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-dns-cert\") pod \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") "
Aug 04 18:34:23.895071 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.894965    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-cacerts\") pod \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") "
Aug 04 18:34:23.895071 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.894997    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-local-certs\") pod \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") "
Aug 04 18:34:23.895071 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.895015    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-token\") pod \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") "
Aug 04 18:34:23.895071 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.895035    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-kubeconfig\") pod \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") "
Aug 04 18:34:23.895298 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.895072    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-ca-configmap\") pod \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") "
Aug 04 18:34:23.895298 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.895091    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8s86\" (UniqueName: \"kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-kube-api-access-h8s86\") pod \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\" (UID: \"f373b7ce-5bd6-4622-8e07-bcf6a825ca10\") "
Aug 04 18:34:23.895543 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.895497    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f373b7ce-5bd6-4622-8e07-bcf6a825ca10" (UID: "f373b7ce-5bd6-4622-8e07-bcf6a825ca10"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:34:23.898097 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.898058    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f373b7ce-5bd6-4622-8e07-bcf6a825ca10" (UID: "f373b7ce-5bd6-4622-8e07-bcf6a825ca10"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:34:23.898097 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.898074    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-local-certs" (OuterVolumeSpecName: "local-certs") pod "f373b7ce-5bd6-4622-8e07-bcf6a825ca10" (UID: "f373b7ce-5bd6-4622-8e07-bcf6a825ca10"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:34:23.898097 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.898083    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-token" (OuterVolumeSpecName: "istio-token") pod "f373b7ce-5bd6-4622-8e07-bcf6a825ca10" (UID: "f373b7ce-5bd6-4622-8e07-bcf6a825ca10"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:34:23.898097 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.898086    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-cacerts" (OuterVolumeSpecName: "cacerts") pod "f373b7ce-5bd6-4622-8e07-bcf6a825ca10" (UID: "f373b7ce-5bd6-4622-8e07-bcf6a825ca10"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:34:23.898336 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.898189    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-kube-api-access-h8s86" (OuterVolumeSpecName: "kube-api-access-h8s86") pod "f373b7ce-5bd6-4622-8e07-bcf6a825ca10" (UID: "f373b7ce-5bd6-4622-8e07-bcf6a825ca10"). InnerVolumeSpecName "kube-api-access-h8s86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:34:23.898336 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.898191    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f373b7ce-5bd6-4622-8e07-bcf6a825ca10" (UID: "f373b7ce-5bd6-4622-8e07-bcf6a825ca10"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:34:23.996776 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.996736    2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-cacerts\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:34:23.996776 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.996775    2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-local-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:34:23.997021 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.996794    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-token\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:34:23.997021 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.996807    2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-kubeconfig\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:34:23.997021 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.996822    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-ca-configmap\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:34:23.997021 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.996835    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8s86\" (UniqueName: \"kubernetes.io/projected/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-kube-api-access-h8s86\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:34:23.997021 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:23.996848    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f373b7ce-5bd6-4622-8e07-bcf6a825ca10-istio-csr-dns-cert\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:34:24.241346 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:24.241261    2578 generic.go:356] "Generic (PLEG): container finished" podID="f373b7ce-5bd6-4622-8e07-bcf6a825ca10" containerID="54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349" exitCode=0
Aug 04 18:34:24.241346 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:24.241315    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"
Aug 04 18:34:24.241604 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:24.241345    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg" event={"ID":"f373b7ce-5bd6-4622-8e07-bcf6a825ca10","Type":"ContainerDied","Data":"54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349"}
Aug 04 18:34:24.241604 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:24.241414    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg" event={"ID":"f373b7ce-5bd6-4622-8e07-bcf6a825ca10","Type":"ContainerDied","Data":"d962384308f5254db530d79264d9981c7852339f8ae47b4661e0732007bfc7d7"}
Aug 04 18:34:24.241604 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:24.241442    2578 scope.go:117] "RemoveContainer" containerID="54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349"
Aug 04 18:34:24.276094 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:24.276047    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"]
Aug 04 18:34:24.283556 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:24.283513    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-75zgg"]
Aug 04 18:34:24.801856 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:24.801821    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f373b7ce-5bd6-4622-8e07-bcf6a825ca10" path="/var/lib/kubelet/pods/f373b7ce-5bd6-4622-8e07-bcf6a825ca10/volumes"
Aug 04 18:34:25.192205 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:25.192110    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-55rcx"
Aug 04 18:34:25.194525 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:25.194498    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-v6srd"
Aug 04 18:34:25.355501 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:25.355433    2578 scope.go:117] "RemoveContainer" containerID="54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349"
Aug 04 18:34:25.355846 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:34:25.355817    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349\": container with ID starting with 54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349 not found: ID does not exist" containerID="54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349"
Aug 04 18:34:25.355928 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:25.355860    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349"} err="failed to get container status \"54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349\": rpc error: code = NotFound desc = could not find container \"54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349\": container with ID starting with 54e566c53f89f3937a573f96b9047659de932a0845f6919277cb195fdde40349 not found: ID does not exist"
Aug 04 18:34:26.252233 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:26.252172    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f" event={"ID":"7c6ca52c-4af7-4bbe-a80b-23c5d1b59064","Type":"ContainerStarted","Data":"a08c94b9e08e96c5acbe7fbbdea8f79bac50b96867a515ce998360dfc3f876c7"}
Aug 04 18:34:26.252679 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:26.252307    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"
Aug 04 18:34:26.254497 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:26.254466    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4" event={"ID":"1f34e173-c23c-401d-ac4d-1606b9139768","Type":"ContainerStarted","Data":"f7571c84ebab1cddf989bbcb19206f785165959a1a7774cb15adf9e3409d69da"}
Aug 04 18:34:26.254643 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:26.254625    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"
Aug 04 18:34:26.285949 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:26.285898    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f" podStartSLOduration=2.229605317 podStartE2EDuration="6.285880502s" totalImagesPullingTime="4.056275185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:34:26.284408298 +0000 UTC m=+408.151323410" watchObservedRunningTime="2026-08-04 18:34:26.285880502 +0000 UTC m=+408.152795603"
Aug 04 18:34:26.326397 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:26.326336    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4" podStartSLOduration=2.901664475 podStartE2EDuration="6.326321023s" totalImagesPullingTime="3.424656548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:34:26.324051659 +0000 UTC m=+408.190966761" watchObservedRunningTime="2026-08-04 18:34:26.326321023 +0000 UTC m=+408.193236124"
Aug 04 18:34:49.170795 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:49.170759    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:49.271275 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:49.271240    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:34:57.260656 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:57.260624    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pd99f"
Aug 04 18:34:57.263646 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:34:57.263623    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-nhls4"
Aug 04 18:35:08.334440 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:08.334345    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:08.496915 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:08.496866    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:25.204995 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:25.204963    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:26.002283 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:26.002238    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:28.574389 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.574348    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-9d264"]
Aug 04 18:35:28.574891 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.574780    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f373b7ce-5bd6-4622-8e07-bcf6a825ca10" containerName="discovery"
Aug 04 18:35:28.574891 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.574799    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f373b7ce-5bd6-4622-8e07-bcf6a825ca10" containerName="discovery"
Aug 04 18:35:28.575008 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.574898    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f373b7ce-5bd6-4622-8e07-bcf6a825ca10" containerName="discovery"
Aug 04 18:35:28.578093 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.578072    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-9d264"
Aug 04 18:35:28.582077 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.582054    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bssm2\""
Aug 04 18:35:28.582440 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.582417    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 04 18:35:28.585735 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.585718    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 04 18:35:28.593830 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.593807    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-9d264"]
Aug 04 18:35:28.644719 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.644681    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnjkx\" (UniqueName: \"kubernetes.io/projected/45e54eea-b7e4-4ea8-b482-41a6e7698942-kube-api-access-pnjkx\") pod \"authorino-operator-68d88948d8-9d264\" (UID: \"45e54eea-b7e4-4ea8-b482-41a6e7698942\") " pod="kuadrant-system/authorino-operator-68d88948d8-9d264"
Aug 04 18:35:28.745597 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.745537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnjkx\" (UniqueName: \"kubernetes.io/projected/45e54eea-b7e4-4ea8-b482-41a6e7698942-kube-api-access-pnjkx\") pod \"authorino-operator-68d88948d8-9d264\" (UID: \"45e54eea-b7e4-4ea8-b482-41a6e7698942\") " pod="kuadrant-system/authorino-operator-68d88948d8-9d264"
Aug 04 18:35:28.762857 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.762826    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnjkx\" (UniqueName: \"kubernetes.io/projected/45e54eea-b7e4-4ea8-b482-41a6e7698942-kube-api-access-pnjkx\") pod \"authorino-operator-68d88948d8-9d264\" (UID: \"45e54eea-b7e4-4ea8-b482-41a6e7698942\") " pod="kuadrant-system/authorino-operator-68d88948d8-9d264"
Aug 04 18:35:28.887756 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:28.887667    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-9d264"
Aug 04 18:35:29.031479 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:29.031403    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-9d264"]
Aug 04 18:35:29.035458 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:35:29.035429    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45e54eea_b7e4_4ea8_b482_41a6e7698942.slice/crio-1f8cb1a64fc5d90d3219e014049878765f0b950f42494746905589e72905d783 WatchSource:0}: Error finding container 1f8cb1a64fc5d90d3219e014049878765f0b950f42494746905589e72905d783: Status 404 returned error can't find the container with id 1f8cb1a64fc5d90d3219e014049878765f0b950f42494746905589e72905d783
Aug 04 18:35:29.146315 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:29.146225    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:29.285034 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:29.285003    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:29.499713 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:29.499628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-9d264" event={"ID":"45e54eea-b7e4-4ea8-b482-41a6e7698942","Type":"ContainerStarted","Data":"1f8cb1a64fc5d90d3219e014049878765f0b950f42494746905589e72905d783"}
Aug 04 18:35:31.031988 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:31.031952    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"]
Aug 04 18:35:31.040762 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:31.040734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:31.047304 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:31.047271    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ftglc\""
Aug 04 18:35:31.053315 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:31.053280    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"]
Aug 04 18:35:31.168686 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:31.168645    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qj8l\" (UniqueName: \"kubernetes.io/projected/4e15ea73-c975-431f-aaf5-948e1889bc44-kube-api-access-9qj8l\") pod \"limitador-operator-controller-manager-6c549b6d85-hx4jx\" (UID: \"4e15ea73-c975-431f-aaf5-948e1889bc44\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:31.270132 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:31.270094    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qj8l\" (UniqueName: \"kubernetes.io/projected/4e15ea73-c975-431f-aaf5-948e1889bc44-kube-api-access-9qj8l\") pod \"limitador-operator-controller-manager-6c549b6d85-hx4jx\" (UID: \"4e15ea73-c975-431f-aaf5-948e1889bc44\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:31.281777 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:31.281749    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qj8l\" (UniqueName: \"kubernetes.io/projected/4e15ea73-c975-431f-aaf5-948e1889bc44-kube-api-access-9qj8l\") pod \"limitador-operator-controller-manager-6c549b6d85-hx4jx\" (UID: \"4e15ea73-c975-431f-aaf5-948e1889bc44\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:31.325790 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:31.325751    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:31.352719 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:31.352690    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:31.418966 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:31.418919    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:31.689638 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:31.689612    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"]
Aug 04 18:35:31.694249 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:35:31.694223    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e15ea73_c975_431f_aaf5_948e1889bc44.slice/crio-6e35344b3feb307067fd7a14ded9d6b159f381df7a15d0ce403de89df4c5e926 WatchSource:0}: Error finding container 6e35344b3feb307067fd7a14ded9d6b159f381df7a15d0ce403de89df4c5e926: Status 404 returned error can't find the container with id 6e35344b3feb307067fd7a14ded9d6b159f381df7a15d0ce403de89df4c5e926
Aug 04 18:35:32.516206 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:32.516166    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx" event={"ID":"4e15ea73-c975-431f-aaf5-948e1889bc44","Type":"ContainerStarted","Data":"6e35344b3feb307067fd7a14ded9d6b159f381df7a15d0ce403de89df4c5e926"}
Aug 04 18:35:32.517795 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:32.517770    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-9d264" event={"ID":"45e54eea-b7e4-4ea8-b482-41a6e7698942","Type":"ContainerStarted","Data":"8199a6bd7a6562b8061d0109830f22075a7978a58a0b20ca6d3bacd03b495d36"}
Aug 04 18:35:32.517934 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:32.517855    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-9d264"
Aug 04 18:35:32.544023 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:32.543973    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-9d264" podStartSLOduration=1.955554739 podStartE2EDuration="4.543955204s" totalImagesPullingTime="2.588400465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:35:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:35:32.542605694 +0000 UTC m=+474.409520797" watchObservedRunningTime="2026-08-04 18:35:32.543955204 +0000 UTC m=+474.410870304"
Aug 04 18:35:32.954595 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:32.954545    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz"]
Aug 04 18:35:32.957911 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:32.957890    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz"
Aug 04 18:35:32.960567 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:32.960542    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7rprz\""
Aug 04 18:35:32.960685 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:32.960542    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Aug 04 18:35:32.978601 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:32.978536    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz"]
Aug 04 18:35:33.084870 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:33.084832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x89q8\" (UniqueName: \"kubernetes.io/projected/bbf8ae72-cbf7-4d33-a481-5c0d6ed2ef7d-kube-api-access-x89q8\") pod \"dns-operator-controller-manager-5985564776-c6dbz\" (UID: \"bbf8ae72-cbf7-4d33-a481-5c0d6ed2ef7d\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz"
Aug 04 18:35:33.185829 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:33.185787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x89q8\" (UniqueName: \"kubernetes.io/projected/bbf8ae72-cbf7-4d33-a481-5c0d6ed2ef7d-kube-api-access-x89q8\") pod \"dns-operator-controller-manager-5985564776-c6dbz\" (UID: \"bbf8ae72-cbf7-4d33-a481-5c0d6ed2ef7d\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz"
Aug 04 18:35:33.200656 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:33.200624    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x89q8\" (UniqueName: \"kubernetes.io/projected/bbf8ae72-cbf7-4d33-a481-5c0d6ed2ef7d-kube-api-access-x89q8\") pod \"dns-operator-controller-manager-5985564776-c6dbz\" (UID: \"bbf8ae72-cbf7-4d33-a481-5c0d6ed2ef7d\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz"
Aug 04 18:35:33.268850 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:33.268781    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz"
Aug 04 18:35:33.574449 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:33.574412    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:33.674826 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:33.674803    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz"]
Aug 04 18:35:33.677835 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:35:33.677810    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf8ae72_cbf7_4d33_a481_5c0d6ed2ef7d.slice/crio-c2ce2edf4389c90583ca5334ad291af771e3753411a1341a29c7cdeaa086bf05 WatchSource:0}: Error finding container c2ce2edf4389c90583ca5334ad291af771e3753411a1341a29c7cdeaa086bf05: Status 404 returned error can't find the container with id c2ce2edf4389c90583ca5334ad291af771e3753411a1341a29c7cdeaa086bf05
Aug 04 18:35:33.807389 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:33.807303    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:34.210331 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:34.210243    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:34.535250 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:34.535149    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz" event={"ID":"bbf8ae72-cbf7-4d33-a481-5c0d6ed2ef7d","Type":"ContainerStarted","Data":"c2ce2edf4389c90583ca5334ad291af771e3753411a1341a29c7cdeaa086bf05"}
Aug 04 18:35:34.536723 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:34.536697    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx" event={"ID":"4e15ea73-c975-431f-aaf5-948e1889bc44","Type":"ContainerStarted","Data":"25e867889bc94b41383a95e2a2caf1c106d7956dd47cf464df06efef2605fb8f"}
Aug 04 18:35:34.536856 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:34.536768    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:34.576088 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:34.576041    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx" podStartSLOduration=1.665470676 podStartE2EDuration="3.576025086s" totalImagesPullingTime="1.91055441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:35:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:35:34.574878408 +0000 UTC m=+476.441793509" watchObservedRunningTime="2026-08-04 18:35:34.576025086 +0000 UTC m=+476.442940187"
Aug 04 18:35:34.646795 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:34.646760    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:35.032836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:35.032805    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:35.299948 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:35.299915    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:36.545937 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:36.545899    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz" event={"ID":"bbf8ae72-cbf7-4d33-a481-5c0d6ed2ef7d","Type":"ContainerStarted","Data":"43053a848df0e4f9c65a8d203a347868001cf8c09a02c4605a8fdb43d1d28078"}
Aug 04 18:35:36.546389 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:36.545982    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz"
Aug 04 18:35:36.614572 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:36.614433    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz" podStartSLOduration=1.907538231 podStartE2EDuration="4.614411762s" totalImagesPullingTime="2.706873531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:35:36.613650715 +0000 UTC m=+478.480565816" watchObservedRunningTime="2026-08-04 18:35:36.614411762 +0000 UTC m=+478.481326864"
Aug 04 18:35:38.028973 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.028937    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"]
Aug 04 18:35:38.032248 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.032230    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:38.035535 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.035512    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j8gx9\""
Aug 04 18:35:38.049514 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.049486    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"]
Aug 04 18:35:38.129776 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.129739    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfbdr\" (UniqueName: \"kubernetes.io/projected/18a64b89-9b19-4f48-9448-afe48fa7b8f3-kube-api-access-tfbdr\") pod \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" (UID: \"18a64b89-9b19-4f48-9448-afe48fa7b8f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:38.129776 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.129784    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18a64b89-9b19-4f48-9448-afe48fa7b8f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" (UID: \"18a64b89-9b19-4f48-9448-afe48fa7b8f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:38.230214 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.230178    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfbdr\" (UniqueName: \"kubernetes.io/projected/18a64b89-9b19-4f48-9448-afe48fa7b8f3-kube-api-access-tfbdr\") pod \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" (UID: \"18a64b89-9b19-4f48-9448-afe48fa7b8f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:38.230389 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.230224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18a64b89-9b19-4f48-9448-afe48fa7b8f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" (UID: \"18a64b89-9b19-4f48-9448-afe48fa7b8f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:38.230685 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.230667    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18a64b89-9b19-4f48-9448-afe48fa7b8f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" (UID: \"18a64b89-9b19-4f48-9448-afe48fa7b8f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:38.246112 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.246087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfbdr\" (UniqueName: \"kubernetes.io/projected/18a64b89-9b19-4f48-9448-afe48fa7b8f3-kube-api-access-tfbdr\") pod \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" (UID: \"18a64b89-9b19-4f48-9448-afe48fa7b8f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:38.342243 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.342213    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:38.484856 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.484799    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"]
Aug 04 18:35:38.487792 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:35:38.487770    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a64b89_9b19_4f48_9448_afe48fa7b8f3.slice/crio-e50baa4856c2a1ccf2d435d35c76c22d22ce575353700e3431feadda11741747 WatchSource:0}: Error finding container e50baa4856c2a1ccf2d435d35c76c22d22ce575353700e3431feadda11741747: Status 404 returned error can't find the container with id e50baa4856c2a1ccf2d435d35c76c22d22ce575353700e3431feadda11741747
Aug 04 18:35:38.555367 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:38.555334    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t" event={"ID":"18a64b89-9b19-4f48-9448-afe48fa7b8f3","Type":"ContainerStarted","Data":"e50baa4856c2a1ccf2d435d35c76c22d22ce575353700e3431feadda11741747"}
Aug 04 18:35:39.436881 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:39.436840    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:39.541951 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:39.541923    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:40.898454 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:40.898397    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:41.083551 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:41.083516    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:41.524699 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:41.524653    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:43.532064 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:43.532033    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-9d264"
Aug 04 18:35:43.617730 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:43.617702    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:43.717473 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:43.717444    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:44.590291 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:44.590256    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t" event={"ID":"18a64b89-9b19-4f48-9448-afe48fa7b8f3","Type":"ContainerStarted","Data":"3da08c19796f91545ccf9980652aef20e5498d17a5ec49a03b08492a984de2ce"}
Aug 04 18:35:44.590699 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:44.590349    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:44.640322 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:44.640266    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t" podStartSLOduration=2.471450321 podStartE2EDuration="7.64024778s" totalImagesPullingTime="5.168797459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:35:44.63678378 +0000 UTC m=+486.503698881" watchObservedRunningTime="2026-08-04 18:35:44.64024778 +0000 UTC m=+486.507162880"
Aug 04 18:35:44.927699 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:44.927622    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:45.036252 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:45.036221    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:45.543287 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:45.543257    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:46.604360 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:46.604328    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:46.771719 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:46.771689    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:47.552055 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:47.552028    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-c6dbz"
Aug 04 18:35:51.391003 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:51.390955    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:51.497728 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:51.497694    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:51.594210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:51.594177    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:55.597154 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:55.597123    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:57.415666 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.415630    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"]
Aug 04 18:35:57.416053 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.415843    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" containerName="manager" containerID="cri-o://3da08c19796f91545ccf9980652aef20e5498d17a5ec49a03b08492a984de2ce" gracePeriod=2
Aug 04 18:35:57.445735 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.445700    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"]
Aug 04 18:35:57.477507 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:57.477467    2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"18a64b89-9b19-4f48-9448-afe48fa7b8f3\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-fq46t\": pods \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:57.480099 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.480072    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"]
Aug 04 18:35:57.480298 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.480269    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" containerName="manager" containerID="cri-o://25e867889bc94b41383a95e2a2caf1c106d7956dd47cf464df06efef2605fb8f" gracePeriod=2
Aug 04 18:35:57.501618 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.501589    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"]
Aug 04 18:35:57.506954 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:57.506922    2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"4e15ea73-c975-431f-aaf5-948e1889bc44\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-6c549b6d85-hx4jx\": pods \"limitador-operator-controller-manager-6c549b6d85-hx4jx\" not found" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:57.510325 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:57.510299    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:57.551280 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.551233    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927"]
Aug 04 18:35:57.551567 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.551555    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" containerName="manager"
Aug 04 18:35:57.551626 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.551569    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" containerName="manager"
Aug 04 18:35:57.551626 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.551604    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" containerName="manager"
Aug 04 18:35:57.551626 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.551610    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" containerName="manager"
Aug 04 18:35:57.551726 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.551658    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" containerName="manager"
Aug 04 18:35:57.551726 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.551667    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" containerName="manager"
Aug 04 18:35:57.554872 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.554853    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927"
Aug 04 18:35:57.558989 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:57.558960    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-hx4jx\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:57.579148 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.579122    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927"]
Aug 04 18:35:57.587042 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.586928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs6x7\" (UniqueName: \"kubernetes.io/projected/7d9a08d1-a47c-4b12-8039-4f482b6d4e5f-kube-api-access-rs6x7\") pod \"limitador-operator-controller-manager-6c549b6d85-7v927\" (UID: \"7d9a08d1-a47c-4b12-8039-4f482b6d4e5f\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927"
Aug 04 18:35:57.609675 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:57.609632    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:57.647801 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.647769    2578 generic.go:356] "Generic (PLEG): container finished" podID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" containerID="3da08c19796f91545ccf9980652aef20e5498d17a5ec49a03b08492a984de2ce" exitCode=0
Aug 04 18:35:57.650221 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.650194    2578 generic.go:356] "Generic (PLEG): container finished" podID="4e15ea73-c975-431f-aaf5-948e1889bc44" containerID="25e867889bc94b41383a95e2a2caf1c106d7956dd47cf464df06efef2605fb8f" exitCode=0
Aug 04 18:35:57.688222 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.688134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs6x7\" (UniqueName: \"kubernetes.io/projected/7d9a08d1-a47c-4b12-8039-4f482b6d4e5f-kube-api-access-rs6x7\") pod \"limitador-operator-controller-manager-6c549b6d85-7v927\" (UID: \"7d9a08d1-a47c-4b12-8039-4f482b6d4e5f\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927"
Aug 04 18:35:57.707504 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.707435    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs6x7\" (UniqueName: \"kubernetes.io/projected/7d9a08d1-a47c-4b12-8039-4f482b6d4e5f-kube-api-access-rs6x7\") pod \"limitador-operator-controller-manager-6c549b6d85-7v927\" (UID: \"7d9a08d1-a47c-4b12-8039-4f482b6d4e5f\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927"
Aug 04 18:35:57.724169 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.724148    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:57.726746 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:57.726718    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:57.726858 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.726843    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:57.729139 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:57.729117    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-hx4jx\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:57.731177 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:57.731156    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-hx4jx\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:57.733836 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:57.733815    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:57.789419 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.789388    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfbdr\" (UniqueName: \"kubernetes.io/projected/18a64b89-9b19-4f48-9448-afe48fa7b8f3-kube-api-access-tfbdr\") pod \"18a64b89-9b19-4f48-9448-afe48fa7b8f3\" (UID: \"18a64b89-9b19-4f48-9448-afe48fa7b8f3\") "
Aug 04 18:35:57.789564 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.789467    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qj8l\" (UniqueName: \"kubernetes.io/projected/4e15ea73-c975-431f-aaf5-948e1889bc44-kube-api-access-9qj8l\") pod \"4e15ea73-c975-431f-aaf5-948e1889bc44\" (UID: \"4e15ea73-c975-431f-aaf5-948e1889bc44\") "
Aug 04 18:35:57.789564 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.789506    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18a64b89-9b19-4f48-9448-afe48fa7b8f3-extensions-socket-volume\") pod \"18a64b89-9b19-4f48-9448-afe48fa7b8f3\" (UID: \"18a64b89-9b19-4f48-9448-afe48fa7b8f3\") "
Aug 04 18:35:57.790022 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.789997    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a64b89-9b19-4f48-9448-afe48fa7b8f3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "18a64b89-9b19-4f48-9448-afe48fa7b8f3" (UID: "18a64b89-9b19-4f48-9448-afe48fa7b8f3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:35:57.791747 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.791728    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18a64b89-9b19-4f48-9448-afe48fa7b8f3-kube-api-access-tfbdr" (OuterVolumeSpecName: "kube-api-access-tfbdr") pod "18a64b89-9b19-4f48-9448-afe48fa7b8f3" (UID: "18a64b89-9b19-4f48-9448-afe48fa7b8f3"). InnerVolumeSpecName "kube-api-access-tfbdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:35:57.791809 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.791756    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e15ea73-c975-431f-aaf5-948e1889bc44-kube-api-access-9qj8l" (OuterVolumeSpecName: "kube-api-access-9qj8l") pod "4e15ea73-c975-431f-aaf5-948e1889bc44" (UID: "4e15ea73-c975-431f-aaf5-948e1889bc44"). InnerVolumeSpecName "kube-api-access-9qj8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:35:57.865109 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.865054    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927"
Aug 04 18:35:57.890843 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.890815    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18a64b89-9b19-4f48-9448-afe48fa7b8f3-extensions-socket-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:35:57.890843 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.890841    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfbdr\" (UniqueName: \"kubernetes.io/projected/18a64b89-9b19-4f48-9448-afe48fa7b8f3-kube-api-access-tfbdr\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:35:57.890843 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:57.890851    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qj8l\" (UniqueName: \"kubernetes.io/projected/4e15ea73-c975-431f-aaf5-948e1889bc44-kube-api-access-9qj8l\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:35:58.012858 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.012833    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927"]
Aug 04 18:35:58.014645 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:35:58.014621    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9a08d1_a47c_4b12_8039_4f482b6d4e5f.slice/crio-ebfdc4d6d2d691b7fbbe784251dfddaa30cabae36d1c90bf655f28d874e5297c WatchSource:0}: Error finding container ebfdc4d6d2d691b7fbbe784251dfddaa30cabae36d1c90bf655f28d874e5297c: Status 404 returned error can't find the container with id ebfdc4d6d2d691b7fbbe784251dfddaa30cabae36d1c90bf655f28d874e5297c
Aug 04 18:35:58.595252 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.595222    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:58.657234 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.657201    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:58.657427 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.657248    2578 scope.go:117] "RemoveContainer" containerID="3da08c19796f91545ccf9980652aef20e5498d17a5ec49a03b08492a984de2ce"
Aug 04 18:35:58.659240 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.659219    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:58.660792 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:58.660667    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:58.661618 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.661130    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927" event={"ID":"7d9a08d1-a47c-4b12-8039-4f482b6d4e5f","Type":"ContainerStarted","Data":"b382c1c4aca4159e3bc42a8f8aaf9047b5a118e25b4f85b7e6da83af563f977b"}
Aug 04 18:35:58.661618 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.661174    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927" event={"ID":"7d9a08d1-a47c-4b12-8039-4f482b6d4e5f","Type":"ContainerStarted","Data":"ebfdc4d6d2d691b7fbbe784251dfddaa30cabae36d1c90bf655f28d874e5297c"}
Aug 04 18:35:58.661618 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.661245    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927"
Aug 04 18:35:58.663719 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:58.663689    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-hx4jx\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:58.666294 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.666277    2578 scope.go:117] "RemoveContainer" containerID="25e867889bc94b41383a95e2a2caf1c106d7956dd47cf464df06efef2605fb8f"
Aug 04 18:35:58.668190 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:58.668142    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:58.671659 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:58.671632    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-hx4jx\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:58.699059 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.698991    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:58.699265 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.699230    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927" podStartSLOduration=1.6992157460000001 podStartE2EDuration="1.699215746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:35:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:35:58.698057619 +0000 UTC m=+500.564972731" watchObservedRunningTime="2026-08-04 18:35:58.699215746 +0000 UTC m=+500.566130847"
Aug 04 18:35:58.700721 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:58.700699    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-hx4jx\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:58.703081 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:58.703059    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:58.732482 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.732455    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"]
Aug 04 18:35:58.737283 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.737267    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:35:58.739490 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:58.739462    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-hx4jx\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-hx4jx"
Aug 04 18:35:58.739827 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.739812    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j8gx9\""
Aug 04 18:35:58.768493 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.768466    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"]
Aug 04 18:35:58.796738 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.796705    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e6f25ea-d775-4787-9037-3acf0b6842b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-7r2k2\" (UID: \"9e6f25ea-d775-4787-9037-3acf0b6842b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:35:58.796917 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.796754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmhpk\" (UniqueName: \"kubernetes.io/projected/9e6f25ea-d775-4787-9037-3acf0b6842b4-kube-api-access-wmhpk\") pod \"kuadrant-operator-controller-manager-68cf589589-7r2k2\" (UID: \"9e6f25ea-d775-4787-9037-3acf0b6842b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:35:58.801786 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.801755    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" path="/var/lib/kubelet/pods/18a64b89-9b19-4f48-9448-afe48fa7b8f3/volumes"
Aug 04 18:35:58.802074 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.802062    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e15ea73-c975-431f-aaf5-948e1889bc44" path="/var/lib/kubelet/pods/4e15ea73-c975-431f-aaf5-948e1889bc44/volumes"
Aug 04 18:35:58.820961 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:35:58.820919    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-fq46t\" is forbidden: User \"system:node:ip-10-0-128-139.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-139.ec2.internal' and this object" podUID="18a64b89-9b19-4f48-9448-afe48fa7b8f3" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-fq46t"
Aug 04 18:35:58.897254 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.897207    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e6f25ea-d775-4787-9037-3acf0b6842b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-7r2k2\" (UID: \"9e6f25ea-d775-4787-9037-3acf0b6842b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:35:58.897430 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.897276    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmhpk\" (UniqueName: \"kubernetes.io/projected/9e6f25ea-d775-4787-9037-3acf0b6842b4-kube-api-access-wmhpk\") pod \"kuadrant-operator-controller-manager-68cf589589-7r2k2\" (UID: \"9e6f25ea-d775-4787-9037-3acf0b6842b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:35:58.897611 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.897567    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e6f25ea-d775-4787-9037-3acf0b6842b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-7r2k2\" (UID: \"9e6f25ea-d775-4787-9037-3acf0b6842b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:35:58.917422 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:58.917390    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmhpk\" (UniqueName: \"kubernetes.io/projected/9e6f25ea-d775-4787-9037-3acf0b6842b4-kube-api-access-wmhpk\") pod \"kuadrant-operator-controller-manager-68cf589589-7r2k2\" (UID: \"9e6f25ea-d775-4787-9037-3acf0b6842b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:35:59.046952 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:59.046923    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:35:59.219005 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:59.218980    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"]
Aug 04 18:35:59.221393 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:35:59.221369    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e6f25ea_d775_4787_9037_3acf0b6842b4.slice/crio-b8e95b19546b6ce7b59714c66bd599cc897638b22d0ec84754da471741481278 WatchSource:0}: Error finding container b8e95b19546b6ce7b59714c66bd599cc897638b22d0ec84754da471741481278: Status 404 returned error can't find the container with id b8e95b19546b6ce7b59714c66bd599cc897638b22d0ec84754da471741481278
Aug 04 18:35:59.537884 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:59.537850    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:59.636998 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:59.636966    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:35:59.669345 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:59.669311    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2" event={"ID":"9e6f25ea-d775-4787-9037-3acf0b6842b4","Type":"ContainerStarted","Data":"812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2"}
Aug 04 18:35:59.669497 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:59.669360    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2" event={"ID":"9e6f25ea-d775-4787-9037-3acf0b6842b4","Type":"ContainerStarted","Data":"b8e95b19546b6ce7b59714c66bd599cc897638b22d0ec84754da471741481278"}
Aug 04 18:35:59.696678 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:35:59.696631    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2" podStartSLOduration=1.69661681 podStartE2EDuration="1.69661681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:35:59.693875114 +0000 UTC m=+501.560790216" watchObservedRunningTime="2026-08-04 18:35:59.69661681 +0000 UTC m=+501.563531910"
Aug 04 18:36:00.673799 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:00.673763    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:36:02.262656 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:02.262624    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:02.361791 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:02.361746    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:05.462001 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:05.461960    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:05.552557 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:05.552523    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:09.671896 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:09.671859    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-7v927"
Aug 04 18:36:11.268188 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:11.268145    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:11.370114 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:11.370081    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:11.438628 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:11.438597    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:11.642310 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:11.642276    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:11.680051 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:11.680028    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:36:28.161746 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.161711    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"]
Aug 04 18:36:28.164846 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.164829    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.167481 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.167462    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7q9c4\""
Aug 04 18:36:28.186106 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.186076    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"]
Aug 04 18:36:28.209173 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.209115    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:28.249778 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.249737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.249778 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.249780    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.249976 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.249838    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.249976 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.249862    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.249976 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.249930    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecb3535a-f51b-4b45-b3f8-03479217df75-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.250071 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.249977    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.250071 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.249996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.250071 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.250016    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.250214 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.250088    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmrc7\" (UniqueName: \"kubernetes.io/projected/ecb3535a-f51b-4b45-b3f8-03479217df75-kube-api-access-hmrc7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.291182 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.291146    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:28.351096 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351059    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmrc7\" (UniqueName: \"kubernetes.io/projected/ecb3535a-f51b-4b45-b3f8-03479217df75-kube-api-access-hmrc7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351264 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351264 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351264 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351264 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351233    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351464 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351265    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecb3535a-f51b-4b45-b3f8-03479217df75-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351464 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351306    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351464 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351334    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351464 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351684 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351497    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351684 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351800 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351724    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.351800 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.351770    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.352109 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.352085    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecb3535a-f51b-4b45-b3f8-03479217df75-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.354077 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.354044    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.354615 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.354594    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.360623 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.360598    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecb3535a-f51b-4b45-b3f8-03479217df75-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.361264 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.361239    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmrc7\" (UniqueName: \"kubernetes.io/projected/ecb3535a-f51b-4b45-b3f8-03479217df75-kube-api-access-hmrc7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xfrfc\" (UID: \"ecb3535a-f51b-4b45-b3f8-03479217df75\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.475239 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.475133    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:28.612356 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.612319    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"]
Aug 04 18:36:28.614658 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:36:28.614628    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecb3535a_f51b_4b45_b3f8_03479217df75.slice/crio-ff1d7c2eeeb864c106543ceb4cbd64f0b5ed9ff3230fb3604e85e79d286fb67d WatchSource:0}: Error finding container ff1d7c2eeeb864c106543ceb4cbd64f0b5ed9ff3230fb3604e85e79d286fb67d: Status 404 returned error can't find the container with id ff1d7c2eeeb864c106543ceb4cbd64f0b5ed9ff3230fb3604e85e79d286fb67d
Aug 04 18:36:28.616625 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.616566    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:36:28.616732 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.616660    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:36:28.616732 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.616690    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:36:28.802910 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.802873    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc" event={"ID":"ecb3535a-f51b-4b45-b3f8-03479217df75","Type":"ContainerStarted","Data":"982728b55e823c28c29bc93f5eae22ab0d25e4c072d7ac7cd56a7ea7a9c0ccc9"}
Aug 04 18:36:28.803087 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.802935    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc" event={"ID":"ecb3535a-f51b-4b45-b3f8-03479217df75","Type":"ContainerStarted","Data":"ff1d7c2eeeb864c106543ceb4cbd64f0b5ed9ff3230fb3604e85e79d286fb67d"}
Aug 04 18:36:28.826525 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:28.826475    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc" podStartSLOduration=0.82645908 podStartE2EDuration="826.45908ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:36:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:36:28.825254295 +0000 UTC m=+530.692169397" watchObservedRunningTime="2026-08-04 18:36:28.82645908 +0000 UTC m=+530.693374181"
Aug 04 18:36:29.476209 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:29.476164    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:29.480820 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:29.480798    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:29.763595 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:29.763488    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:29.803658 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:29.803627    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:29.804754 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:29.804733    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:36:29.886158 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:29.886126    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:29.952731 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:29.952700    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:30.693127 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:30.693084    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:30.793199 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:30.793114    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:34.643427 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:34.643392    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:50.977646 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:50.977615    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:51.075999 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:51.075958    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:36:51.461326 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:36:51.461286    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:03.514086 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:03.514052    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:03.614316 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:03.614273    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:09.983847 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:09.983809    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:10.090728 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:10.090694    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:10.162754 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:10.162723    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:13.556280 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:13.556225    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:13.654257 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:13.654222    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:22.623372 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:22.623334    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:42.168955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:42.168917    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:42.266453 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:42.266423    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:43.677547 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:43.677506    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:43.857432 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:43.857380    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:44.439241 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.439212    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:44.534812 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.534777    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:44.605874 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.605844    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"]
Aug 04 18:37:44.608125 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.608109    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"
Aug 04 18:37:44.610939 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.610919    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Aug 04 18:37:44.611863 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.611843    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-d4xk8\""
Aug 04 18:37:44.617638 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.617618    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"]
Aug 04 18:37:44.678937 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.678905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d427g\" (UniqueName: \"kubernetes.io/projected/10355037-0234-4030-a93d-eccdcba6252b-kube-api-access-d427g\") pod \"payload-pre-processing-687dbb6cf6-lt4zt\" (UID: \"10355037-0234-4030-a93d-eccdcba6252b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"
Aug 04 18:37:44.679342 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.678950    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10355037-0234-4030-a93d-eccdcba6252b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-lt4zt\" (UID: \"10355037-0234-4030-a93d-eccdcba6252b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"
Aug 04 18:37:44.780472 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.780379    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d427g\" (UniqueName: \"kubernetes.io/projected/10355037-0234-4030-a93d-eccdcba6252b-kube-api-access-d427g\") pod \"payload-pre-processing-687dbb6cf6-lt4zt\" (UID: \"10355037-0234-4030-a93d-eccdcba6252b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"
Aug 04 18:37:44.780472 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.780447    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10355037-0234-4030-a93d-eccdcba6252b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-lt4zt\" (UID: \"10355037-0234-4030-a93d-eccdcba6252b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"
Aug 04 18:37:44.781315 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.781291    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10355037-0234-4030-a93d-eccdcba6252b-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-lt4zt\" (UID: \"10355037-0234-4030-a93d-eccdcba6252b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"
Aug 04 18:37:44.790129 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.790104    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d427g\" (UniqueName: \"kubernetes.io/projected/10355037-0234-4030-a93d-eccdcba6252b-kube-api-access-d427g\") pod \"payload-pre-processing-687dbb6cf6-lt4zt\" (UID: \"10355037-0234-4030-a93d-eccdcba6252b\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"
Aug 04 18:37:44.919116 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:44.919081    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"
Aug 04 18:37:45.067248 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:45.067215    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"]
Aug 04 18:37:45.070410 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:37:45.070381    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10355037_0234_4030_a93d_eccdcba6252b.slice/crio-54d8f3889734ccf511c3eb4a43177f4379c62a225e46e8b8d529745312ee990a WatchSource:0}: Error finding container 54d8f3889734ccf511c3eb4a43177f4379c62a225e46e8b8d529745312ee990a: Status 404 returned error can't find the container with id 54d8f3889734ccf511c3eb4a43177f4379c62a225e46e8b8d529745312ee990a
Aug 04 18:37:45.129426 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:45.129393    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt" event={"ID":"10355037-0234-4030-a93d-eccdcba6252b","Type":"ContainerStarted","Data":"54d8f3889734ccf511c3eb4a43177f4379c62a225e46e8b8d529745312ee990a"}
Aug 04 18:37:47.145141 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:47.145021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt" event={"ID":"10355037-0234-4030-a93d-eccdcba6252b","Type":"ContainerStarted","Data":"2e82237f39ad9fbf784558824a49aa0f93d653258962796f9bc2d2ecfc14569c"}
Aug 04 18:37:47.145141 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:47.145113    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"
Aug 04 18:37:47.175748 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:47.175699    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt" podStartSLOduration=1.437357783 podStartE2EDuration="3.17568323s" totalImagesPullingTime="1.738325447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:37:47.175470364 +0000 UTC m=+609.042385465" watchObservedRunningTime="2026-08-04 18:37:47.17568323 +0000 UTC m=+609.042598331"
Aug 04 18:37:54.177955 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:54.177921    2578 generic.go:356] "Generic (PLEG): container finished" podID="ecb3535a-f51b-4b45-b3f8-03479217df75" containerID="982728b55e823c28c29bc93f5eae22ab0d25e4c072d7ac7cd56a7ea7a9c0ccc9" exitCode=0
Aug 04 18:37:54.178398 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:54.177990    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc" event={"ID":"ecb3535a-f51b-4b45-b3f8-03479217df75","Type":"ContainerDied","Data":"982728b55e823c28c29bc93f5eae22ab0d25e4c072d7ac7cd56a7ea7a9c0ccc9"}
Aug 04 18:37:54.178398 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:54.178347    2578 scope.go:117] "RemoveContainer" containerID="982728b55e823c28c29bc93f5eae22ab0d25e4c072d7ac7cd56a7ea7a9c0ccc9"
Aug 04 18:37:54.179224 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:54.179197    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:37:54.179286 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:54.179261    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:37:54.179324 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:54.179293    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:37:55.184160 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:55.184121    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc" event={"ID":"ecb3535a-f51b-4b45-b3f8-03479217df75","Type":"ContainerStarted","Data":"067dc9534a8c33a37344854f0fa6ce931df1325adf6bd60da36927c8b286a6db"}
Aug 04 18:37:55.364661 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:55.364627    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:55.467681 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:55.467599    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:37:55.475513 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:55.475495    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:37:55.476504 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:55.476479    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xfrfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Aug 04 18:37:55.476613 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:55.476544    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc" podUID="ecb3535a-f51b-4b45-b3f8-03479217df75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Aug 04 18:37:56.475760 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:56.475729    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xfrfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Aug 04 18:37:56.476129 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:56.475781    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc" podUID="ecb3535a-f51b-4b45-b3f8-03479217df75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Aug 04 18:37:57.476080 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:57.476048    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xfrfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Aug 04 18:37:57.476488 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:57.476101    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc" podUID="ecb3535a-f51b-4b45-b3f8-03479217df75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Aug 04 18:37:58.151598 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:58.151543    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-lt4zt"
Aug 04 18:37:58.475871 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:58.475784    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xfrfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Aug 04 18:37:58.475871 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:58.475837    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc" podUID="ecb3535a-f51b-4b45-b3f8-03479217df75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Aug 04 18:37:59.475883 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:59.475853    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xfrfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Aug 04 18:37:59.476266 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:59.475905    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc" podUID="ecb3535a-f51b-4b45-b3f8-03479217df75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Aug 04 18:37:59.804024 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:37:59.803988    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:38:00.480187 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:00.480153    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:38:00.481142 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:00.481121    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xfrfc"
Aug 04 18:38:02.851626 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:02.851593    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:38:07.483636 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.483601    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"]
Aug 04 18:38:07.486990 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.486970    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.489765 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.489747    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 04 18:38:07.489842 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.489788    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Aug 04 18:38:07.490787 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.490768    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 04 18:38:07.490878 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.490854    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fqx86\""
Aug 04 18:38:07.499508 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.499485    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"]
Aug 04 18:38:07.552452 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.552418    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:38:07.599424 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.599390    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50921b6e-45e6-43cb-8b39-7a66c75cea95-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.599628 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.599453    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.599628 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.599498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.599628 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.599519    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.599628 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.599605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.599775 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.599643    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmjm9\" (UniqueName: \"kubernetes.io/projected/50921b6e-45e6-43cb-8b39-7a66c75cea95-kube-api-access-fmjm9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.700868 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.700829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.701063 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.700880    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.701063 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.700896    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.701063 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.700922    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.701063 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.700947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmjm9\" (UniqueName: \"kubernetes.io/projected/50921b6e-45e6-43cb-8b39-7a66c75cea95-kube-api-access-fmjm9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.701063 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.700978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50921b6e-45e6-43cb-8b39-7a66c75cea95-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.701350 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.701318    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.701350 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.701334    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.701498 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.701388    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.703070 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.703053    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50921b6e-45e6-43cb-8b39-7a66c75cea95-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.703485 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.703470    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50921b6e-45e6-43cb-8b39-7a66c75cea95-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.709931 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.709908    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmjm9\" (UniqueName: \"kubernetes.io/projected/50921b6e-45e6-43cb-8b39-7a66c75cea95-kube-api-access-fmjm9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vqp4r\" (UID: \"50921b6e-45e6-43cb-8b39-7a66c75cea95\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.796459 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.796427    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:07.931615 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:07.931523    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"]
Aug 04 18:38:07.934008 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:38:07.933977    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50921b6e_45e6_43cb_8b39_7a66c75cea95.slice/crio-5f0ccaff02333e455df653c5501f25269f5e4b0cb4ca1a8eb6350b726bdd5fa3 WatchSource:0}: Error finding container 5f0ccaff02333e455df653c5501f25269f5e4b0cb4ca1a8eb6350b726bdd5fa3: Status 404 returned error can't find the container with id 5f0ccaff02333e455df653c5501f25269f5e4b0cb4ca1a8eb6350b726bdd5fa3
Aug 04 18:38:08.242830 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:08.242746    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r" event={"ID":"50921b6e-45e6-43cb-8b39-7a66c75cea95","Type":"ContainerStarted","Data":"5f0ccaff02333e455df653c5501f25269f5e4b0cb4ca1a8eb6350b726bdd5fa3"}
Aug 04 18:38:09.248838 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:09.248754    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r" event={"ID":"50921b6e-45e6-43cb-8b39-7a66c75cea95","Type":"ContainerStarted","Data":"1bdb21622f1b68c577ae71f6599a3e8e87dd61021ccf92442c1325701c7e95d5"}
Aug 04 18:38:09.275805 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:09.275761    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r" podStartSLOduration=1.296838679 podStartE2EDuration="2.275744148s" totalImagesPullingTime="978.905469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:38:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:38:09.274679066 +0000 UTC m=+631.141594159" watchObservedRunningTime="2026-08-04 18:38:09.275744148 +0000 UTC m=+631.142659295"
Aug 04 18:38:17.797019 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:17.796975    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:17.797707 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:17.797082    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:17.809388 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:17.809363    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:18.299401 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:18.299375    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vqp4r"
Aug 04 18:38:32.368783 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.368751    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"]
Aug 04 18:38:32.374511 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.374495    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.378308 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.378292    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Aug 04 18:38:32.385862 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.385842    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"]
Aug 04 18:38:32.400954 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.400933    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:38:32.418347 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.418325    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.418460 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.418362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.418460 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.418432    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c19e1e5-851a-4399-826d-90bed88da3aa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.418460 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.418455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.418569 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.418488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsss8\" (UniqueName: \"kubernetes.io/projected/7c19e1e5-851a-4399-826d-90bed88da3aa-kube-api-access-fsss8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.418569 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.418516    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.518943 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.518911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsss8\" (UniqueName: \"kubernetes.io/projected/7c19e1e5-851a-4399-826d-90bed88da3aa-kube-api-access-fsss8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.519097 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.518953    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.519155 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.519130    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.519214 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.519199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.519289 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.519267    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.519289 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.519280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c19e1e5-851a-4399-826d-90bed88da3aa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.519440 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.519308    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.519526 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.519501    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.519630 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.519607    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.521648 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.521625    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c19e1e5-851a-4399-826d-90bed88da3aa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.521763 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.521736    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c19e1e5-851a-4399-826d-90bed88da3aa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.529332 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.529181    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsss8\" (UniqueName: \"kubernetes.io/projected/7c19e1e5-851a-4399-826d-90bed88da3aa-kube-api-access-fsss8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bxxlc\" (UID: \"7c19e1e5-851a-4399-826d-90bed88da3aa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.684817 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.684724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:32.817290 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.817233    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"]
Aug 04 18:38:32.820015 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:38:32.819986    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c19e1e5_851a_4399_826d_90bed88da3aa.slice/crio-a05332220233158ac39afeaacfba18e4033abf1c219925583e1793c1ed18aaa4 WatchSource:0}: Error finding container a05332220233158ac39afeaacfba18e4033abf1c219925583e1793c1ed18aaa4: Status 404 returned error can't find the container with id a05332220233158ac39afeaacfba18e4033abf1c219925583e1793c1ed18aaa4
Aug 04 18:38:32.821181 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:32.821160    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 18:38:33.354237 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:33.354203    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc" event={"ID":"7c19e1e5-851a-4399-826d-90bed88da3aa","Type":"ContainerStarted","Data":"d1ffb3736512ad2260ddf075431ab33ca74a5ee5852eda0521e01936db4eb697"}
Aug 04 18:38:33.354398 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:33.354256    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc" event={"ID":"7c19e1e5-851a-4399-826d-90bed88da3aa","Type":"ContainerStarted","Data":"a05332220233158ac39afeaacfba18e4033abf1c219925583e1793c1ed18aaa4"}
Aug 04 18:38:33.375647 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:33.375532    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc" podStartSLOduration=1.079207062 podStartE2EDuration="1.375513304s" totalImagesPullingTime="296.306242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:38:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:38:33.37467764 +0000 UTC m=+655.241592740" watchObservedRunningTime="2026-08-04 18:38:33.375513304 +0000 UTC m=+655.242428428"
Aug 04 18:38:42.685087 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:42.685047    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:42.685695 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:42.685160    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:42.697401 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:42.697372    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:43.407721 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:43.407693    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bxxlc"
Aug 04 18:38:43.598333 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:43.598300    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:38:52.571194 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:38:52.571154    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:03.988695 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:03.988662    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:04.056455 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.056424    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:04.582499 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.582468    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"]
Aug 04 18:39:04.585791 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.585774    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.588386 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.588367    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Aug 04 18:39:04.597834 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.597812    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"]
Aug 04 18:39:04.600528 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.600504    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.600655 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.600542    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a98801e-320a-48fd-bde6-fa151e154182-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.600655 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.600567    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.600764 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.600669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x68gj\" (UniqueName: \"kubernetes.io/projected/1a98801e-320a-48fd-bde6-fa151e154182-kube-api-access-x68gj\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.600764 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.600697    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.600764 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.600740    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.609077 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.609050    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:04.701635 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.701604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x68gj\" (UniqueName: \"kubernetes.io/projected/1a98801e-320a-48fd-bde6-fa151e154182-kube-api-access-x68gj\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.701798 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.701650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.701798 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.701700    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.701798 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.701730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.701798 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.701766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a98801e-320a-48fd-bde6-fa151e154182-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.701995 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.701802    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.702136 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.702107    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.702243 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.702228    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.702302 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.702228    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.703991 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.703970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a98801e-320a-48fd-bde6-fa151e154182-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.704293 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.704275    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a98801e-320a-48fd-bde6-fa151e154182-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.714046 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.714017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x68gj\" (UniqueName: \"kubernetes.io/projected/1a98801e-320a-48fd-bde6-fa151e154182-kube-api-access-x68gj\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cgllg\" (UID: \"1a98801e-320a-48fd-bde6-fa151e154182\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:04.894287 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:04.894215    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:05.019027 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:05.018973    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"]
Aug 04 18:39:05.021489 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:39:05.021464    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a98801e_320a_48fd_bde6_fa151e154182.slice/crio-1e36f26e1058fee6578525896d36dc1cdf421e02cc6f7e1082c4a22d13bc250b WatchSource:0}: Error finding container 1e36f26e1058fee6578525896d36dc1cdf421e02cc6f7e1082c4a22d13bc250b: Status 404 returned error can't find the container with id 1e36f26e1058fee6578525896d36dc1cdf421e02cc6f7e1082c4a22d13bc250b
Aug 04 18:39:05.503332 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:05.503294    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg" event={"ID":"1a98801e-320a-48fd-bde6-fa151e154182","Type":"ContainerStarted","Data":"21c2639f3fc3be8dccea35133c18c02904342825935c6e27e8bae7da88ae98ca"}
Aug 04 18:39:05.503332 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:05.503338    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg" event={"ID":"1a98801e-320a-48fd-bde6-fa151e154182","Type":"ContainerStarted","Data":"1e36f26e1058fee6578525896d36dc1cdf421e02cc6f7e1082c4a22d13bc250b"}
Aug 04 18:39:05.524557 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:05.524457    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg" podStartSLOduration=1.280424264 podStartE2EDuration="1.524442368s" totalImagesPullingTime="244.018104ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:39:05.522377977 +0000 UTC m=+687.389293075" watchObservedRunningTime="2026-08-04 18:39:05.524442368 +0000 UTC m=+687.391357536"
Aug 04 18:39:14.895084 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:14.895047    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:14.895084 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:14.895089    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:14.907336 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:14.907311    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:15.566265 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:15.566236    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cgllg"
Aug 04 18:39:19.824643 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:19.824599    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:19.925893 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:19.925860    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:54.802932 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:54.802875    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:54.903492 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:54.903459    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:55.394216 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:55.394182    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:55.496644 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:55.496604    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:55.695205 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:55.695127    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:55.828609 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:55.828560    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:39:55.934765 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:39:55.934737    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:40:02.036190 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:40:02.036156    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:40:06.452204 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:40:06.452164    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:40:06.518022 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:40:06.518001    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:40:07.544648 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:40:07.544618    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:40:07.637920 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:40:07.637884    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:41:47.543225 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:41:47.543171    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:42:00.053355 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:42:00.053305    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:42:01.337726 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:42:01.337689    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:43:02.858119 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:43:02.858085    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:43:02.941927 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:43:02.941899    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:43:03.106650 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:43:03.106613    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:43:09.419173 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:43:09.419140    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:43:09.534908 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:43:09.534877    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:43:09.981210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:43:09.981175    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:43:10.091994 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:43:10.091962    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:43:15.932253 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:43:15.932220    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:43:16.007442 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:43:16.007406    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:47:34.026552 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:47:34.026478    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:48:32.403032 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:32.402951    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"]
Aug 04 18:48:32.403508 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:32.403158    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2" podUID="9e6f25ea-d775-4787-9037-3acf0b6842b4" containerName="manager" containerID="cri-o://812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2" gracePeriod=10
Aug 04 18:48:32.657384 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:32.657330    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:48:32.741295 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:32.741269    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e6f25ea-d775-4787-9037-3acf0b6842b4-extensions-socket-volume\") pod \"9e6f25ea-d775-4787-9037-3acf0b6842b4\" (UID: \"9e6f25ea-d775-4787-9037-3acf0b6842b4\") "
Aug 04 18:48:32.741451 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:32.741317    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmhpk\" (UniqueName: \"kubernetes.io/projected/9e6f25ea-d775-4787-9037-3acf0b6842b4-kube-api-access-wmhpk\") pod \"9e6f25ea-d775-4787-9037-3acf0b6842b4\" (UID: \"9e6f25ea-d775-4787-9037-3acf0b6842b4\") "
Aug 04 18:48:32.741604 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:32.741560    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e6f25ea-d775-4787-9037-3acf0b6842b4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9e6f25ea-d775-4787-9037-3acf0b6842b4" (UID: "9e6f25ea-d775-4787-9037-3acf0b6842b4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:48:32.743223 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:32.743200    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e6f25ea-d775-4787-9037-3acf0b6842b4-kube-api-access-wmhpk" (OuterVolumeSpecName: "kube-api-access-wmhpk") pod "9e6f25ea-d775-4787-9037-3acf0b6842b4" (UID: "9e6f25ea-d775-4787-9037-3acf0b6842b4"). InnerVolumeSpecName "kube-api-access-wmhpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:48:32.842126 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:32.842104    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmhpk\" (UniqueName: \"kubernetes.io/projected/9e6f25ea-d775-4787-9037-3acf0b6842b4-kube-api-access-wmhpk\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:48:32.842126 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:32.842126    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e6f25ea-d775-4787-9037-3acf0b6842b4-extensions-socket-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:48:33.009619 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:33.009540    2578 generic.go:356] "Generic (PLEG): container finished" podID="9e6f25ea-d775-4787-9037-3acf0b6842b4" containerID="812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2" exitCode=0
Aug 04 18:48:33.009619 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:33.009603    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"
Aug 04 18:48:33.009758 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:33.009628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2" event={"ID":"9e6f25ea-d775-4787-9037-3acf0b6842b4","Type":"ContainerDied","Data":"812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2"}
Aug 04 18:48:33.009758 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:33.009679    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2" event={"ID":"9e6f25ea-d775-4787-9037-3acf0b6842b4","Type":"ContainerDied","Data":"b8e95b19546b6ce7b59714c66bd599cc897638b22d0ec84754da471741481278"}
Aug 04 18:48:33.009758 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:33.009701    2578 scope.go:117] "RemoveContainer" containerID="812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2"
Aug 04 18:48:33.018683 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:33.018666    2578 scope.go:117] "RemoveContainer" containerID="812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2"
Aug 04 18:48:33.018907 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:48:33.018883    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2\": container with ID starting with 812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2 not found: ID does not exist" containerID="812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2"
Aug 04 18:48:33.018959 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:33.018916    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2"} err="failed to get container status \"812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2\": rpc error: code = NotFound desc = could not find container \"812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2\": container with ID starting with 812e2e8429c8e7def54de224b56fd19f065bd0f4d7c747dfd9a645cfde1d61b2 not found: ID does not exist"
Aug 04 18:48:33.031868 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:33.031846    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"]
Aug 04 18:48:33.036842 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:33.036818    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-7r2k2"]
Aug 04 18:48:34.802009 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:48:34.801977    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e6f25ea-d775-4787-9037-3acf0b6842b4" path="/var/lib/kubelet/pods/9e6f25ea-d775-4787-9037-3acf0b6842b4/volumes"
Aug 04 18:49:33.079729 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.079691    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"]
Aug 04 18:49:33.080124 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.080019    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e6f25ea-d775-4787-9037-3acf0b6842b4" containerName="manager"
Aug 04 18:49:33.080124 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.080030    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e6f25ea-d775-4787-9037-3acf0b6842b4" containerName="manager"
Aug 04 18:49:33.080124 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.080096    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e6f25ea-d775-4787-9037-3acf0b6842b4" containerName="manager"
Aug 04 18:49:33.082990 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.082974    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 18:49:33.085667 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.085643    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j8gx9\""
Aug 04 18:49:33.094936 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.094919    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"]
Aug 04 18:49:33.126400 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.126370    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prsvg\" (UniqueName: \"kubernetes.io/projected/d7d0caee-a593-42c1-9dd9-3723b03244ed-kube-api-access-prsvg\") pod \"kuadrant-operator-controller-manager-68cf589589-g4xpg\" (UID: \"d7d0caee-a593-42c1-9dd9-3723b03244ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 18:49:33.126541 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.126433    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d0caee-a593-42c1-9dd9-3723b03244ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-g4xpg\" (UID: \"d7d0caee-a593-42c1-9dd9-3723b03244ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 18:49:33.227374 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.227347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prsvg\" (UniqueName: \"kubernetes.io/projected/d7d0caee-a593-42c1-9dd9-3723b03244ed-kube-api-access-prsvg\") pod \"kuadrant-operator-controller-manager-68cf589589-g4xpg\" (UID: \"d7d0caee-a593-42c1-9dd9-3723b03244ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 18:49:33.227499 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.227403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d0caee-a593-42c1-9dd9-3723b03244ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-g4xpg\" (UID: \"d7d0caee-a593-42c1-9dd9-3723b03244ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 18:49:33.227758 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.227741    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d0caee-a593-42c1-9dd9-3723b03244ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-g4xpg\" (UID: \"d7d0caee-a593-42c1-9dd9-3723b03244ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 18:49:33.236084 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.236056    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prsvg\" (UniqueName: \"kubernetes.io/projected/d7d0caee-a593-42c1-9dd9-3723b03244ed-kube-api-access-prsvg\") pod \"kuadrant-operator-controller-manager-68cf589589-g4xpg\" (UID: \"d7d0caee-a593-42c1-9dd9-3723b03244ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 18:49:33.392340 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.392273    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 18:49:33.535500 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:33.535478    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"]
Aug 04 18:49:33.538292 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:49:33.538262    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d0caee_a593_42c1_9dd9_3723b03244ed.slice/crio-08f11cfc96916903c6b136d14236e35ae0ce69f6487c8caa05ad5e35b0a523bf WatchSource:0}: Error finding container 08f11cfc96916903c6b136d14236e35ae0ce69f6487c8caa05ad5e35b0a523bf: Status 404 returned error can't find the container with id 08f11cfc96916903c6b136d14236e35ae0ce69f6487c8caa05ad5e35b0a523bf
Aug 04 18:49:34.282522 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:34.282483    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg" event={"ID":"d7d0caee-a593-42c1-9dd9-3723b03244ed","Type":"ContainerStarted","Data":"4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03"}
Aug 04 18:49:34.282931 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:34.282605    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg" event={"ID":"d7d0caee-a593-42c1-9dd9-3723b03244ed","Type":"ContainerStarted","Data":"08f11cfc96916903c6b136d14236e35ae0ce69f6487c8caa05ad5e35b0a523bf"}
Aug 04 18:49:34.282931 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:34.282638    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 18:49:34.313470 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:34.313420    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg" podStartSLOduration=1.313404155 podStartE2EDuration="1.313404155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:49:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:49:34.312305524 +0000 UTC m=+1316.179220626" watchObservedRunningTime="2026-08-04 18:49:34.313404155 +0000 UTC m=+1316.180319254"
Aug 04 18:49:45.184498 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:45.184460    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:49:45.289631 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:45.289601    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 18:49:47.442047 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:49:47.442008    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:52:14.492656 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:14.492622    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:52:35.170118 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:35.170084    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:52:35.257018 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:35.256988    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:52:42.880356 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:42.880320    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"]
Aug 04 18:52:42.884092 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:42.884069    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:42.886568 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:42.886545    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-849990f8-kserve-self-signed-certs\""
Aug 04 18:52:42.906799 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:42.906771    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"]
Aug 04 18:52:42.923834 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:42.923796    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:52:43.061216 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.061178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c4ck\" (UniqueName: \"kubernetes.io/projected/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-kube-api-access-6c4ck\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.061383 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.061229    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tls-certs\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.061383 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.061305    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-dshm\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.061383 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.061345    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-model-cache\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.061383 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.061364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-home\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.061383 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.061378    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tmp-dir\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.162762 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.162687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c4ck\" (UniqueName: \"kubernetes.io/projected/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-kube-api-access-6c4ck\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.162762 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.162738    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tls-certs\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.162963 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.162779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-dshm\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.162963 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.162818    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-model-cache\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.162963 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.162836    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-home\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.162963 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.162857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tmp-dir\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.163281 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.163255    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-home\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.163378 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.163309    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-model-cache\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.163378 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.163346    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tmp-dir\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.165011 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.164986    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-dshm\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.165252 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.165234    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tls-certs\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.170890 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.170869    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c4ck\" (UniqueName: \"kubernetes.io/projected/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-kube-api-access-6c4ck\") pod \"e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") " pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.194857 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.194836    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:43.324054 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:43.323986    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"]
Aug 04 18:52:43.326734 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:52:43.326698    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20fe9c10_f8bb_4ee4_a0fe_d45c0c120cef.slice/crio-9621fb74f33e6e03d9320464198c76973e7aa712f6803f1ec28a2c042b6aba03 WatchSource:0}: Error finding container 9621fb74f33e6e03d9320464198c76973e7aa712f6803f1ec28a2c042b6aba03: Status 404 returned error can't find the container with id 9621fb74f33e6e03d9320464198c76973e7aa712f6803f1ec28a2c042b6aba03
Aug 04 18:52:44.129728 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:44.129695    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4" event={"ID":"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef","Type":"ContainerStarted","Data":"22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598"}
Aug 04 18:52:44.129728 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:44.129733    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4" event={"ID":"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef","Type":"ContainerStarted","Data":"9621fb74f33e6e03d9320464198c76973e7aa712f6803f1ec28a2c042b6aba03"}
Aug 04 18:52:44.150054 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:44.150004    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4" podStartSLOduration=2.1499886 podStartE2EDuration="2.1499886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:52:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:52:44.14820981 +0000 UTC m=+1506.015124910" watchObservedRunningTime="2026-08-04 18:52:44.1499886 +0000 UTC m=+1506.016903727"
Aug 04 18:52:53.195356 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:53.195267    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:53.195356 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:53.195316    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:53.208127 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:53.208097    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:54.188177 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:54.188143    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:52:58.688354 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.688320    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"]
Aug 04 18:52:58.691921 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.691904    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.696400 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.696379    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-849990f8-kserve-self-signed-certs\""
Aug 04 18:52:58.705132 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.705106    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"]
Aug 04 18:52:58.738134 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.738099    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:52:58.808141 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.808093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-dshm\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.808354 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.808161    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-home\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.808354 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.808248    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-model-cache\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.808354 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.808305    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73cc6537-e4b1-4037-8e35-49a94a88fe26-tls-certs\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.808354 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.808343    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8vfw\" (UniqueName: \"kubernetes.io/projected/73cc6537-e4b1-4037-8e35-49a94a88fe26-kube-api-access-t8vfw\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.808516 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.808426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-tmp-dir\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.909300 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.909249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8vfw\" (UniqueName: \"kubernetes.io/projected/73cc6537-e4b1-4037-8e35-49a94a88fe26-kube-api-access-t8vfw\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.909503 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.909333    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-tmp-dir\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.909503 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.909366    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-dshm\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.909503 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.909416    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-home\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.909503 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.909467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-model-cache\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.909788 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.909509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73cc6537-e4b1-4037-8e35-49a94a88fe26-tls-certs\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.909851 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.909828    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-tmp-dir\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.909904 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.909879    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-home\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.909940 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.909923    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-model-cache\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.911828 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.911801    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-dshm\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.912025 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.912002    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73cc6537-e4b1-4037-8e35-49a94a88fe26-tls-certs\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:58.919361 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:58.919332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8vfw\" (UniqueName: \"kubernetes.io/projected/73cc6537-e4b1-4037-8e35-49a94a88fe26-kube-api-access-t8vfw\") pod \"e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") " pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:59.001224 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:59.001138    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:52:59.143249 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:59.143218    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"]
Aug 04 18:52:59.144456 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:52:59.144429    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73cc6537_e4b1_4037_8e35_49a94a88fe26.slice/crio-19a3e91ebf2ad8cb5d52b259599929fd567180712a24f0090bc41eb70023eca0 WatchSource:0}: Error finding container 19a3e91ebf2ad8cb5d52b259599929fd567180712a24f0090bc41eb70023eca0: Status 404 returned error can't find the container with id 19a3e91ebf2ad8cb5d52b259599929fd567180712a24f0090bc41eb70023eca0
Aug 04 18:52:59.204539 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:52:59.204509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w" event={"ID":"73cc6537-e4b1-4037-8e35-49a94a88fe26","Type":"ContainerStarted","Data":"19a3e91ebf2ad8cb5d52b259599929fd567180712a24f0090bc41eb70023eca0"}
Aug 04 18:53:00.211084 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:00.211047    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w" event={"ID":"73cc6537-e4b1-4037-8e35-49a94a88fe26","Type":"ContainerStarted","Data":"1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3"}
Aug 04 18:53:00.232940 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:00.232880    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w" podStartSLOduration=2.23285943 podStartE2EDuration="2.23285943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:52:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:53:00.232072668 +0000 UTC m=+1522.098987763" watchObservedRunningTime="2026-08-04 18:53:00.23285943 +0000 UTC m=+1522.099774534"
Aug 04 18:53:09.001602 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:09.001494    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:53:09.001602 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:09.001548    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:53:09.013831 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:09.013805    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:53:09.268178 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:09.268096    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:53:36.884765 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:36.884725    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"]
Aug 04 18:53:38.377446 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:38.377410    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:53:39.100805 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:39.100774    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"]
Aug 04 18:53:39.175671 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:39.175634    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:53:51.903444 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:51.903379    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4" podUID="20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" containerName="main" containerID="cri-o://22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598" gracePeriod=45
Aug 04 18:53:52.149156 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.149134    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:53:52.171507 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.171437    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-dshm\") pod \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") "
Aug 04 18:53:52.171507 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.171502    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6c4ck\" (UniqueName: \"kubernetes.io/projected/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-kube-api-access-6c4ck\") pod \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") "
Aug 04 18:53:52.171742 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.171543    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-model-cache\") pod \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") "
Aug 04 18:53:52.171742 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.171614    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tls-certs\") pod \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") "
Aug 04 18:53:52.171742 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.171666    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-home\") pod \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") "
Aug 04 18:53:52.171742 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.171701    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tmp-dir\") pod \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\" (UID: \"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef\") "
Aug 04 18:53:52.171954 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.171887    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-model-cache" (OuterVolumeSpecName: "model-cache") pod "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" (UID: "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:53:52.172105 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.172055    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-model-cache\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:52.172105 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.172052    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-home" (OuterVolumeSpecName: "home") pod "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" (UID: "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:53:52.172105 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.172067    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" (UID: "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:53:52.174435 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.174373    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" (UID: "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:53:52.174435 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.174401    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-dshm" (OuterVolumeSpecName: "dshm") pod "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" (UID: "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:53:52.174624 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.174468    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-kube-api-access-6c4ck" (OuterVolumeSpecName: "kube-api-access-6c4ck") pod "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" (UID: "20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef"). InnerVolumeSpecName "kube-api-access-6c4ck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:53:52.272665 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.272635    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-home\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:52.272665 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.272658    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tmp-dir\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:52.272665 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.272668    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-dshm\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:52.272894 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.272678    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6c4ck\" (UniqueName: \"kubernetes.io/projected/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-kube-api-access-6c4ck\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:52.272894 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.272687    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef-tls-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:52.465788 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.465699    2578 generic.go:356] "Generic (PLEG): container finished" podID="20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" containerID="22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598" exitCode=0
Aug 04 18:53:52.465788 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.465751    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"
Aug 04 18:53:52.465957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.465786    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4" event={"ID":"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef","Type":"ContainerDied","Data":"22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598"}
Aug 04 18:53:52.465957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.465830    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4" event={"ID":"20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef","Type":"ContainerDied","Data":"9621fb74f33e6e03d9320464198c76973e7aa712f6803f1ec28a2c042b6aba03"}
Aug 04 18:53:52.465957 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.465852    2578 scope.go:117] "RemoveContainer" containerID="22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598"
Aug 04 18:53:52.475003 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.474984    2578 scope.go:117] "RemoveContainer" containerID="22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598"
Aug 04 18:53:52.475283 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:53:52.475264    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598\": container with ID starting with 22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598 not found: ID does not exist" containerID="22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598"
Aug 04 18:53:52.475336 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.475292    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598"} err="failed to get container status \"22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598\": rpc error: code = NotFound desc = could not find container \"22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598\": container with ID starting with 22d3f82fc66c81ad74c13c3cd5cfae10980ba79a19bc77b30803af1abf86d598 not found: ID does not exist"
Aug 04 18:53:52.487681 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.487658    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"]
Aug 04 18:53:52.494111 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.494089    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-849990f8-kserve-759c5b6967-nz8s4"]
Aug 04 18:53:52.801901 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:52.801872    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" path="/var/lib/kubelet/pods/20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef/volumes"
Aug 04 18:53:54.121635 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.121597    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w" podUID="73cc6537-e4b1-4037-8e35-49a94a88fe26" containerName="main" containerID="cri-o://1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3" gracePeriod=45
Aug 04 18:53:54.374924 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.374872    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:53:54.477020 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.476987    2578 generic.go:356] "Generic (PLEG): container finished" podID="73cc6537-e4b1-4037-8e35-49a94a88fe26" containerID="1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3" exitCode=0
Aug 04 18:53:54.477020 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.477024    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w" event={"ID":"73cc6537-e4b1-4037-8e35-49a94a88fe26","Type":"ContainerDied","Data":"1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3"}
Aug 04 18:53:54.477267 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.477044    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"
Aug 04 18:53:54.477267 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.477058    2578 scope.go:117] "RemoveContainer" containerID="1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3"
Aug 04 18:53:54.477267 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.477050    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w" event={"ID":"73cc6537-e4b1-4037-8e35-49a94a88fe26","Type":"ContainerDied","Data":"19a3e91ebf2ad8cb5d52b259599929fd567180712a24f0090bc41eb70023eca0"}
Aug 04 18:53:54.487744 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.487727    2578 scope.go:117] "RemoveContainer" containerID="1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3"
Aug 04 18:53:54.488026 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:53:54.487997    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3\": container with ID starting with 1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3 not found: ID does not exist" containerID="1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3"
Aug 04 18:53:54.488130 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.488028    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3"} err="failed to get container status \"1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3\": rpc error: code = NotFound desc = could not find container \"1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3\": container with ID starting with 1d39d10a4d3c92eeafa083a20faed8802e38ce15d1b0e4d95f056416767995d3 not found: ID does not exist"
Aug 04 18:53:54.490816 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.490797    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8vfw\" (UniqueName: \"kubernetes.io/projected/73cc6537-e4b1-4037-8e35-49a94a88fe26-kube-api-access-t8vfw\") pod \"73cc6537-e4b1-4037-8e35-49a94a88fe26\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") "
Aug 04 18:53:54.490869 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.490829    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73cc6537-e4b1-4037-8e35-49a94a88fe26-tls-certs\") pod \"73cc6537-e4b1-4037-8e35-49a94a88fe26\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") "
Aug 04 18:53:54.490908 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.490888    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-model-cache\") pod \"73cc6537-e4b1-4037-8e35-49a94a88fe26\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") "
Aug 04 18:53:54.490949 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.490916    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-dshm\") pod \"73cc6537-e4b1-4037-8e35-49a94a88fe26\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") "
Aug 04 18:53:54.490984 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.490964    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-home\") pod \"73cc6537-e4b1-4037-8e35-49a94a88fe26\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") "
Aug 04 18:53:54.491029 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.490989    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-tmp-dir\") pod \"73cc6537-e4b1-4037-8e35-49a94a88fe26\" (UID: \"73cc6537-e4b1-4037-8e35-49a94a88fe26\") "
Aug 04 18:53:54.491183 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.491145    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-model-cache" (OuterVolumeSpecName: "model-cache") pod "73cc6537-e4b1-4037-8e35-49a94a88fe26" (UID: "73cc6537-e4b1-4037-8e35-49a94a88fe26"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:53:54.491327 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.491209    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-home" (OuterVolumeSpecName: "home") pod "73cc6537-e4b1-4037-8e35-49a94a88fe26" (UID: "73cc6537-e4b1-4037-8e35-49a94a88fe26"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:53:54.491429 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.491400    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "73cc6537-e4b1-4037-8e35-49a94a88fe26" (UID: "73cc6537-e4b1-4037-8e35-49a94a88fe26"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:53:54.493227 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.493203    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-dshm" (OuterVolumeSpecName: "dshm") pod "73cc6537-e4b1-4037-8e35-49a94a88fe26" (UID: "73cc6537-e4b1-4037-8e35-49a94a88fe26"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:53:54.493335 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.493315    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73cc6537-e4b1-4037-8e35-49a94a88fe26-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "73cc6537-e4b1-4037-8e35-49a94a88fe26" (UID: "73cc6537-e4b1-4037-8e35-49a94a88fe26"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 18:53:54.493499 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.493463    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73cc6537-e4b1-4037-8e35-49a94a88fe26-kube-api-access-t8vfw" (OuterVolumeSpecName: "kube-api-access-t8vfw") pod "73cc6537-e4b1-4037-8e35-49a94a88fe26" (UID: "73cc6537-e4b1-4037-8e35-49a94a88fe26"). InnerVolumeSpecName "kube-api-access-t8vfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:53:54.591711 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.591680    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8vfw\" (UniqueName: \"kubernetes.io/projected/73cc6537-e4b1-4037-8e35-49a94a88fe26-kube-api-access-t8vfw\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:54.591711 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.591706    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73cc6537-e4b1-4037-8e35-49a94a88fe26-tls-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:54.591711 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.591716    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-model-cache\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:54.591930 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.591723    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-dshm\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:54.591930 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.591731    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-home\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:54.591930 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.591738    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73cc6537-e4b1-4037-8e35-49a94a88fe26-tmp-dir\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:53:54.803315 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.803287    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"]
Aug 04 18:53:54.804735 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:54.804716    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-849990f8-kserve-8565b55b6c-85f2w"]
Aug 04 18:53:56.802267 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:53:56.802235    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73cc6537-e4b1-4037-8e35-49a94a88fe26" path="/var/lib/kubelet/pods/73cc6537-e4b1-4037-8e35-49a94a88fe26/volumes"
Aug 04 18:54:57.583469 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:54:57.583432    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:56:35.120673 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:56:35.120562    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:00.319089 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:00.319037    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:26.190675 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:26.190639    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:29.146019 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:29.145991    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:29.304175 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:29.304143    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:31.512119 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:31.512034    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:36.380418 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:36.380381    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:36.472188 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:36.472152    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:36.607607 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:36.607561    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:36.946268 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:36.946229    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:37.295072 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:37.295038    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:44.638528 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:44.638478    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:46.399215 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.399182    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:46.589596 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.589547    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:46.745443 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.745368    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:57:46.790751 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.790721    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"]
Aug 04 18:57:46.791077 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.791064    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" containerName="main"
Aug 04 18:57:46.791137 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.791079    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" containerName="main"
Aug 04 18:57:46.791137 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.791096    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73cc6537-e4b1-4037-8e35-49a94a88fe26" containerName="main"
Aug 04 18:57:46.791137 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.791101    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="73cc6537-e4b1-4037-8e35-49a94a88fe26" containerName="main"
Aug 04 18:57:46.791247 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.791172    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="20fe9c10-f8bb-4ee4-a0fe-d45c0c120cef" containerName="main"
Aug 04 18:57:46.791247 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.791181    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="73cc6537-e4b1-4037-8e35-49a94a88fe26" containerName="main"
Aug 04 18:57:46.793774 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.793757    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:57:46.796272 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.796238    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-93cfcc2b\""
Aug 04 18:57:46.796562 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.796542    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-93cfcc2b-dockercfg-rpm2x\""
Aug 04 18:57:46.814357 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.814335    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"]
Aug 04 18:57:46.921696 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.921661    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9lbs\" (UniqueName: \"kubernetes.io/projected/74480216-f0a6-4883-899e-656c13a0d9e1-kube-api-access-f9lbs\") pod \"payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq\" (UID: \"74480216-f0a6-4883-899e-656c13a0d9e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:57:46.921872 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:46.921711    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74480216-f0a6-4883-899e-656c13a0d9e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq\" (UID: \"74480216-f0a6-4883-899e-656c13a0d9e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:57:47.023185 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:47.023115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9lbs\" (UniqueName: \"kubernetes.io/projected/74480216-f0a6-4883-899e-656c13a0d9e1-kube-api-access-f9lbs\") pod \"payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq\" (UID: \"74480216-f0a6-4883-899e-656c13a0d9e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:57:47.023185 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:47.023157    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74480216-f0a6-4883-899e-656c13a0d9e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq\" (UID: \"74480216-f0a6-4883-899e-656c13a0d9e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:57:47.023869 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:47.023851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74480216-f0a6-4883-899e-656c13a0d9e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq\" (UID: \"74480216-f0a6-4883-899e-656c13a0d9e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:57:47.037735 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:47.037712    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9lbs\" (UniqueName: \"kubernetes.io/projected/74480216-f0a6-4883-899e-656c13a0d9e1-kube-api-access-f9lbs\") pod \"payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq\" (UID: \"74480216-f0a6-4883-899e-656c13a0d9e1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:57:47.104848 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:47.104823    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:57:47.257562 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:47.257532    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"]
Aug 04 18:57:47.261069 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:57:47.261026    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74480216_f0a6_4883_899e_656c13a0d9e1.slice/crio-4827d21f89acf9425c3a843547e995af142d3e69b21974261e98d2a1ef1fb8b4 WatchSource:0}: Error finding container 4827d21f89acf9425c3a843547e995af142d3e69b21974261e98d2a1ef1fb8b4: Status 404 returned error can't find the container with id 4827d21f89acf9425c3a843547e995af142d3e69b21974261e98d2a1ef1fb8b4
Aug 04 18:57:47.538392 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:47.538365    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq" event={"ID":"74480216-f0a6-4883-899e-656c13a0d9e1","Type":"ContainerStarted","Data":"bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5"}
Aug 04 18:57:47.538781 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:47.538444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq" event={"ID":"74480216-f0a6-4883-899e-656c13a0d9e1","Type":"ContainerStarted","Data":"4827d21f89acf9425c3a843547e995af142d3e69b21974261e98d2a1ef1fb8b4"}
Aug 04 18:57:47.538781 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:47.538515    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:57:58.545394 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:58.545362    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:57:58.564093 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:57:58.564049    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq" podStartSLOduration=12.564034434 podStartE2EDuration="12.564034434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:57:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:57:47.56511806 +0000 UTC m=+1809.432033161" watchObservedRunningTime="2026-08-04 18:57:58.564034434 +0000 UTC m=+1820.430949577"
Aug 04 18:58:16.328425 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.328396    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:16.366650 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.366616    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"]
Aug 04 18:58:16.366887 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.366854    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq" podUID="74480216-f0a6-4883-899e-656c13a0d9e1" containerName="payload-pre-processing" containerID="cri-o://bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5" gracePeriod=30
Aug 04 18:58:16.433764 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.433731    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:16.534625 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.534380    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:16.609943 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.609918    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:58:16.669879 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.669833    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:16.675765 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.675726    2578 generic.go:356] "Generic (PLEG): container finished" podID="74480216-f0a6-4883-899e-656c13a0d9e1" containerID="bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5" exitCode=0
Aug 04 18:58:16.675915 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.675803    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"
Aug 04 18:58:16.675915 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.675855    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq" event={"ID":"74480216-f0a6-4883-899e-656c13a0d9e1","Type":"ContainerDied","Data":"bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5"}
Aug 04 18:58:16.675915 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.675906    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq" event={"ID":"74480216-f0a6-4883-899e-656c13a0d9e1","Type":"ContainerDied","Data":"4827d21f89acf9425c3a843547e995af142d3e69b21974261e98d2a1ef1fb8b4"}
Aug 04 18:58:16.676083 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.675940    2578 scope.go:117] "RemoveContainer" containerID="bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5"
Aug 04 18:58:16.689702 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.689686    2578 scope.go:117] "RemoveContainer" containerID="bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5"
Aug 04 18:58:16.690045 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:58:16.690007    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5\": container with ID starting with bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5 not found: ID does not exist" containerID="bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5"
Aug 04 18:58:16.690135 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.690049    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5"} err="failed to get container status \"bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5\": rpc error: code = NotFound desc = could not find container \"bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5\": container with ID starting with bc16bef68a16baa950873fa61fbb5c53c30edc7511ce7f71e321a7e639e429d5 not found: ID does not exist"
Aug 04 18:58:16.771054 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.771031    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9lbs\" (UniqueName: \"kubernetes.io/projected/74480216-f0a6-4883-899e-656c13a0d9e1-kube-api-access-f9lbs\") pod \"74480216-f0a6-4883-899e-656c13a0d9e1\" (UID: \"74480216-f0a6-4883-899e-656c13a0d9e1\") "
Aug 04 18:58:16.771178 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.771066    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74480216-f0a6-4883-899e-656c13a0d9e1-plugins-config-volume\") pod \"74480216-f0a6-4883-899e-656c13a0d9e1\" (UID: \"74480216-f0a6-4883-899e-656c13a0d9e1\") "
Aug 04 18:58:16.771438 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.771415    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74480216-f0a6-4883-899e-656c13a0d9e1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "74480216-f0a6-4883-899e-656c13a0d9e1" (UID: "74480216-f0a6-4883-899e-656c13a0d9e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:58:16.773036 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.773020    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74480216-f0a6-4883-899e-656c13a0d9e1-kube-api-access-f9lbs" (OuterVolumeSpecName: "kube-api-access-f9lbs") pod "74480216-f0a6-4883-899e-656c13a0d9e1" (UID: "74480216-f0a6-4883-899e-656c13a0d9e1"). InnerVolumeSpecName "kube-api-access-f9lbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:58:16.823217 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.823185    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:16.872598 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.872495    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9lbs\" (UniqueName: \"kubernetes.io/projected/74480216-f0a6-4883-899e-656c13a0d9e1-kube-api-access-f9lbs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:16.872598 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.872522    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74480216-f0a6-4883-899e-656c13a0d9e1-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:16.924255 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.924218    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:16.993428 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:16.993398    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"]
Aug 04 18:58:17.000283 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.000262    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93cfcc2b-64dfd7545b-f4pbq"]
Aug 04 18:58:17.262490 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.262398    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:17.707116 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.707085    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"]
Aug 04 18:58:17.707458 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.707405    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74480216-f0a6-4883-899e-656c13a0d9e1" containerName="payload-pre-processing"
Aug 04 18:58:17.707458 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.707415    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="74480216-f0a6-4883-899e-656c13a0d9e1" containerName="payload-pre-processing"
Aug 04 18:58:17.707528 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.707472    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="74480216-f0a6-4883-899e-656c13a0d9e1" containerName="payload-pre-processing"
Aug 04 18:58:17.711718 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.711698    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.714454 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.714427    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-2646b40b-gw-openshift-default-dockercfg-td7bb\""
Aug 04 18:58:17.732603 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.732565    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"]
Aug 04 18:58:17.880948 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.880918    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-certs\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.880948 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.880949    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-data\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.881156 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.880971    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-credential-socket\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.881156 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.880993    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-envoy\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.881156 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.881024    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-token\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.881156 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.881083    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-socket\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.881156 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.881120    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-podinfo\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.881315 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.881164    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxkzk\" (UniqueName: \"kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-kube-api-access-xxkzk\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.881315 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.881187    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83517cba-548c-4784-9db9-6a1cf3a2c29a-istiod-ca-cert\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.982284 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.982217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-podinfo\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.982284 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.982264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxkzk\" (UniqueName: \"kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-kube-api-access-xxkzk\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.982530 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.982507    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83517cba-548c-4784-9db9-6a1cf3a2c29a-istiod-ca-cert\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.982623 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.982602    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-certs\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.982705 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.982641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-data\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.982705 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.982681    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-credential-socket\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.982818 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.982719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-envoy\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.982818 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.982743    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-token\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.982954 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.982928    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-socket\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.983045 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.983027    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-data\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.983114 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.983086    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-credential-socket\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.983328 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.983303    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83517cba-548c-4784-9db9-6a1cf3a2c29a-istiod-ca-cert\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.983400 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.983324    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-certs\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.983400 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.983379    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-socket\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.985018 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.984997    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-podinfo\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.985210 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.985190    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-envoy\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.990922 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.990902    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-token\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:17.990922 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:17.990910    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxkzk\" (UniqueName: \"kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-kube-api-access-xxkzk\") pod \"e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") " pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:18.022474 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:18.022451    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:18.167328 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:18.167306    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"]
Aug 04 18:58:18.169448 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:58:18.169419    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83517cba_548c_4784_9db9_6a1cf3a2c29a.slice/crio-61934d05c3fc8e49c7ddd229c521dacba5d8efd95cd4c6025ed642f24f703fc1 WatchSource:0}: Error finding container 61934d05c3fc8e49c7ddd229c521dacba5d8efd95cd4c6025ed642f24f703fc1: Status 404 returned error can't find the container with id 61934d05c3fc8e49c7ddd229c521dacba5d8efd95cd4c6025ed642f24f703fc1
Aug 04 18:58:18.171891 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:18.171858    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:58:18.171975 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:18.171923    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:58:18.171975 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:18.171951    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:58:18.688659 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:18.688625    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" event={"ID":"83517cba-548c-4784-9db9-6a1cf3a2c29a","Type":"ContainerStarted","Data":"cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720"}
Aug 04 18:58:18.688847 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:18.688676    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" event={"ID":"83517cba-548c-4784-9db9-6a1cf3a2c29a","Type":"ContainerStarted","Data":"61934d05c3fc8e49c7ddd229c521dacba5d8efd95cd4c6025ed642f24f703fc1"}
Aug 04 18:58:18.709783 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:18.709733    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" podStartSLOduration=1.709715074 podStartE2EDuration="1.709715074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:58:18.708970023 +0000 UTC m=+1840.575885125" watchObservedRunningTime="2026-08-04 18:58:18.709715074 +0000 UTC m=+1840.576630228"
Aug 04 18:58:18.802749 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:18.802719    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74480216-f0a6-4883-899e-656c13a0d9e1" path="/var/lib/kubelet/pods/74480216-f0a6-4883-899e-656c13a0d9e1/volumes"
Aug 04 18:58:19.023701 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:19.023626    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:19.028158 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:19.028126    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:19.107441 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:19.107412    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:19.209060 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:19.209034    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:19.693299 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:19.693273    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:19.694277 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:19.694259    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:20.181234 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:20.181199    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:20.562877 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:20.562846    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:20.940953 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:20.940877    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:21.046735 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.046701    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:21.205637 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.205538    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"]
Aug 04 18:58:21.209151 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.209134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:21.211820 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.211794    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-2646b40b\""
Aug 04 18:58:21.212403 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.212369    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-2646b40b-dockercfg-5bbf2\""
Aug 04 18:58:21.212719 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.212695    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:21.222759 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.222738    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"]
Aug 04 18:58:21.313641 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.313608    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzzpl\" (UniqueName: \"kubernetes.io/projected/26621fe9-7a7c-4314-981d-01e84b4a4c9d-kube-api-access-wzzpl\") pod \"payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh\" (UID: \"26621fe9-7a7c-4314-981d-01e84b4a4c9d\") " pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:21.313846 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.313734    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26621fe9-7a7c-4314-981d-01e84b4a4c9d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh\" (UID: \"26621fe9-7a7c-4314-981d-01e84b4a4c9d\") " pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:21.414623 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.414560    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26621fe9-7a7c-4314-981d-01e84b4a4c9d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh\" (UID: \"26621fe9-7a7c-4314-981d-01e84b4a4c9d\") " pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:21.414803 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.414673    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzzpl\" (UniqueName: \"kubernetes.io/projected/26621fe9-7a7c-4314-981d-01e84b4a4c9d-kube-api-access-wzzpl\") pod \"payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh\" (UID: \"26621fe9-7a7c-4314-981d-01e84b4a4c9d\") " pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:21.415509 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.415484    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26621fe9-7a7c-4314-981d-01e84b4a4c9d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh\" (UID: \"26621fe9-7a7c-4314-981d-01e84b4a4c9d\") " pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:21.423290 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.423254    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzzpl\" (UniqueName: \"kubernetes.io/projected/26621fe9-7a7c-4314-981d-01e84b4a4c9d-kube-api-access-wzzpl\") pod \"payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh\" (UID: \"26621fe9-7a7c-4314-981d-01e84b4a4c9d\") " pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:21.520720 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.520627    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:21.656860 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.656837    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"]
Aug 04 18:58:21.657489 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:58:21.657462    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26621fe9_7a7c_4314_981d_01e84b4a4c9d.slice/crio-fadcf0cfe9e1d90d00c52ca3ec7e30db9584f2874eb9c6d443ee7ed666129f47 WatchSource:0}: Error finding container fadcf0cfe9e1d90d00c52ca3ec7e30db9584f2874eb9c6d443ee7ed666129f47: Status 404 returned error can't find the container with id fadcf0cfe9e1d90d00c52ca3ec7e30db9584f2874eb9c6d443ee7ed666129f47
Aug 04 18:58:21.702629 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:21.702603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh" event={"ID":"26621fe9-7a7c-4314-981d-01e84b4a4c9d","Type":"ContainerStarted","Data":"fadcf0cfe9e1d90d00c52ca3ec7e30db9584f2874eb9c6d443ee7ed666129f47"}
Aug 04 18:58:22.708791 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:22.708756    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh" event={"ID":"26621fe9-7a7c-4314-981d-01e84b4a4c9d","Type":"ContainerStarted","Data":"6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd"}
Aug 04 18:58:22.709173 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:22.708825    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:22.724836 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:22.724790    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh" podStartSLOduration=1.7247726939999999 podStartE2EDuration="1.724772694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:58:22.72440171 +0000 UTC m=+1844.591316811" watchObservedRunningTime="2026-08-04 18:58:22.724772694 +0000 UTC m=+1844.591687795"
Aug 04 18:58:32.768740 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:32.768708    2578 generic.go:356] "Generic (PLEG): container finished" podID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerID="cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720" exitCode=0
Aug 04 18:58:32.769158 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:32.768765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" event={"ID":"83517cba-548c-4784-9db9-6a1cf3a2c29a","Type":"ContainerDied","Data":"cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720"}
Aug 04 18:58:32.769158 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:32.769141    2578 scope.go:117] "RemoveContainer" containerID="cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720"
Aug 04 18:58:32.770219 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:32.770188    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:58:32.770294 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:32.770274    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:58:32.770340 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:32.770317    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:58:33.718887 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:33.718841    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:33.776454 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:33.776422    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" event={"ID":"83517cba-548c-4784-9db9-6a1cf3a2c29a","Type":"ContainerStarted","Data":"add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c"}
Aug 04 18:58:34.023036 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:34.022940    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:34.024424 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:34.024378    2578 patch_prober.go:28] interesting pod/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Aug 04 18:58:34.024569 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:34.024455    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Aug 04 18:58:34.693235 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:34.693203    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:35.023443 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:35.023342    2578 patch_prober.go:28] interesting pod/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Aug 04 18:58:35.023443 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:35.023421    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Aug 04 18:58:36.023231 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:36.023192    2578 patch_prober.go:28] interesting pod/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Aug 04 18:58:36.023451 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:36.023269    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Aug 04 18:58:37.024040 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:37.023997    2578 patch_prober.go:28] interesting pod/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Aug 04 18:58:37.024496 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:37.024070    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Aug 04 18:58:38.023243 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:38.022795    2578 patch_prober.go:28] interesting pod/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Aug 04 18:58:38.023243 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:38.022859    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Aug 04 18:58:39.027415 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:39.027341    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:39.028255 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:39.028236    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:42.195715 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:42.195676    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:51.829928 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:51.829898    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:51.868318 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:51.868289    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"]
Aug 04 18:58:51.868550 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:51.868524    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh" podUID="26621fe9-7a7c-4314-981d-01e84b4a4c9d" containerName="payload-processing" containerID="cri-o://6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd" gracePeriod=30
Aug 04 18:58:51.923570 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:51.923542    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:52.015192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.015160    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:52.094895 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.094813    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:52.123023 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.122986    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:52.196437 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.194779    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzzpl\" (UniqueName: \"kubernetes.io/projected/26621fe9-7a7c-4314-981d-01e84b4a4c9d-kube-api-access-wzzpl\") pod \"26621fe9-7a7c-4314-981d-01e84b4a4c9d\" (UID: \"26621fe9-7a7c-4314-981d-01e84b4a4c9d\") "
Aug 04 18:58:52.196437 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.194888    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26621fe9-7a7c-4314-981d-01e84b4a4c9d-plugins-config-volume\") pod \"26621fe9-7a7c-4314-981d-01e84b4a4c9d\" (UID: \"26621fe9-7a7c-4314-981d-01e84b4a4c9d\") "
Aug 04 18:58:52.196437 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.195957    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26621fe9-7a7c-4314-981d-01e84b4a4c9d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26621fe9-7a7c-4314-981d-01e84b4a4c9d" (UID: "26621fe9-7a7c-4314-981d-01e84b4a4c9d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:58:52.199521 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.199494    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26621fe9-7a7c-4314-981d-01e84b4a4c9d-kube-api-access-wzzpl" (OuterVolumeSpecName: "kube-api-access-wzzpl") pod "26621fe9-7a7c-4314-981d-01e84b4a4c9d" (UID: "26621fe9-7a7c-4314-981d-01e84b4a4c9d"). InnerVolumeSpecName "kube-api-access-wzzpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:58:52.205638 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.205050    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"]
Aug 04 18:58:52.205638 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.205299    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy" containerID="cri-o://add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c" gracePeriod=30
Aug 04 18:58:52.230336 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.230304    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:52.296028 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.295995    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzzpl\" (UniqueName: \"kubernetes.io/projected/26621fe9-7a7c-4314-981d-01e84b4a4c9d-kube-api-access-wzzpl\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:52.296028 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.296027    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26621fe9-7a7c-4314-981d-01e84b4a4c9d-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:52.456960 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.456880    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:52.668172 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.668141    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:52.867297 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.867267    2578 generic.go:356] "Generic (PLEG): container finished" podID="26621fe9-7a7c-4314-981d-01e84b4a4c9d" containerID="6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd" exitCode=0
Aug 04 18:58:52.867748 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.867319    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"
Aug 04 18:58:52.867748 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.867318    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh" event={"ID":"26621fe9-7a7c-4314-981d-01e84b4a4c9d","Type":"ContainerDied","Data":"6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd"}
Aug 04 18:58:52.867748 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.867553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh" event={"ID":"26621fe9-7a7c-4314-981d-01e84b4a4c9d","Type":"ContainerDied","Data":"fadcf0cfe9e1d90d00c52ca3ec7e30db9584f2874eb9c6d443ee7ed666129f47"}
Aug 04 18:58:52.867748 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.867609    2578 scope.go:117] "RemoveContainer" containerID="6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd"
Aug 04 18:58:52.875842 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.875653    2578 scope.go:117] "RemoveContainer" containerID="6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd"
Aug 04 18:58:52.875930 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:58:52.875915    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd\": container with ID starting with 6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd not found: ID does not exist" containerID="6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd"
Aug 04 18:58:52.875971 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.875945    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd"} err="failed to get container status \"6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd\": rpc error: code = NotFound desc = could not find container \"6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd\": container with ID starting with 6e2739fc80ecd2a52538246443d29d27470f3c4ad350d759299ab8778b2218bd not found: ID does not exist"
Aug 04 18:58:52.908130 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.908105    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"]
Aug 04 18:58:52.919795 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.919774    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2646b40b-6658b46845-fc4xh"]
Aug 04 18:58:52.998909 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.998860    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"]
Aug 04 18:58:52.999285 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.999271    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26621fe9-7a7c-4314-981d-01e84b4a4c9d" containerName="payload-processing"
Aug 04 18:58:52.999285 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.999285    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="26621fe9-7a7c-4314-981d-01e84b4a4c9d" containerName="payload-processing"
Aug 04 18:58:52.999398 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:52.999344    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="26621fe9-7a7c-4314-981d-01e84b4a4c9d" containerName="payload-processing"
Aug 04 18:58:53.003690 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.003666    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.006820 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.006801    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-41669ddd-openshift-default-dockercfg-pd94g\""
Aug 04 18:58:53.029830 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.029804    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"]
Aug 04 18:58:53.058534 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.058476    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:53.103324 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.103295    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istiod-ca-cert\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.103478 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.103343    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-socket\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.103478 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.103369    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-podinfo\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.103478 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.103462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-credential-socket\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.103698 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.103508    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-data\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.103698 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.103570    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-token\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.103698 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.103627    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg6jm\" (UniqueName: \"kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-kube-api-access-hg6jm\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.103867 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.103703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-envoy\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.103867 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.103767    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-certs\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.204612 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.204557    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istiod-ca-cert\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.204612 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.204616    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-socket\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.204849 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.204634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-podinfo\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.204849 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.204668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-credential-socket\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.204849 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.204688    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-data\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.204849 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.204706    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-token\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.204849 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.204720    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg6jm\" (UniqueName: \"kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-kube-api-access-hg6jm\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.204849 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.204764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-envoy\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.204849 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.204800    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-certs\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.205189 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.205111    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-data\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.205254 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.205197    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-certs\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.205254 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.205221    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-credential-socket\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.205254 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.205232    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-socket\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.205407 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.205389    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istiod-ca-cert\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.207078 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.207058    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-envoy\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.207175 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.207109    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-podinfo\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.213991 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.213971    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-token\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.214172 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.214151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg6jm\" (UniqueName: \"kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-kube-api-access-hg6jm\") pod \"e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") " pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.314015 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.313925    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:53.466966 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.466820    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"]
Aug 04 18:58:53.468980 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:58:53.468955    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca5f0a0_cfb9_4d25_949e_8cb4957e9e11.slice/crio-8c57524166b0f6abd04868e0322f069bdce2df1f37c348327d5ea276683bb830 WatchSource:0}: Error finding container 8c57524166b0f6abd04868e0322f069bdce2df1f37c348327d5ea276683bb830: Status 404 returned error can't find the container with id 8c57524166b0f6abd04868e0322f069bdce2df1f37c348327d5ea276683bb830
Aug 04 18:58:53.470214 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.470189    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:53.471027 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.470998    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:58:53.471125 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.471066    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:58:53.471125 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.471110    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:58:53.873380 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.873313    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" event={"ID":"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11","Type":"ContainerStarted","Data":"68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2"}
Aug 04 18:58:53.873845 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.873424    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" event={"ID":"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11","Type":"ContainerStarted","Data":"8c57524166b0f6abd04868e0322f069bdce2df1f37c348327d5ea276683bb830"}
Aug 04 18:58:53.897549 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:53.897507    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" podStartSLOduration=1.8974931430000002 podStartE2EDuration="1.897493143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:58:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:58:53.895823788 +0000 UTC m=+1875.762738900" watchObservedRunningTime="2026-08-04 18:58:53.897493143 +0000 UTC m=+1875.764408244"
Aug 04 18:58:54.030309 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:54.030274    2578 patch_prober.go:28] interesting pod/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 18:58:54.030446 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:54.030341    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 18:58:54.314387 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:54.314343    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:54.319855 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:54.319827    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:54.465438 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:54.465401    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:54.803238 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:54.803197    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26621fe9-7a7c-4314-981d-01e84b4a4c9d" path="/var/lib/kubelet/pods/26621fe9-7a7c-4314-981d-01e84b4a4c9d/volumes"
Aug 04 18:58:54.879871 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:54.879842    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:54.880847 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:54.880828    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:58:55.284981 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:55.284897    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:55.484239 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:55.484209    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:56.525396 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:56.525365    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:56.832770 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:56.832733    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:57.472696 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.472675    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:57.647680 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.647649    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-credential-socket\") pod \"83517cba-548c-4784-9db9-6a1cf3a2c29a\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") "
Aug 04 18:58:57.648149 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.647691    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-podinfo\") pod \"83517cba-548c-4784-9db9-6a1cf3a2c29a\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") "
Aug 04 18:58:57.648149 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.647717    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxkzk\" (UniqueName: \"kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-kube-api-access-xxkzk\") pod \"83517cba-548c-4784-9db9-6a1cf3a2c29a\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") "
Aug 04 18:58:57.648149 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.647757    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-envoy\") pod \"83517cba-548c-4784-9db9-6a1cf3a2c29a\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") "
Aug 04 18:58:57.648149 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.647782    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-data\") pod \"83517cba-548c-4784-9db9-6a1cf3a2c29a\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") "
Aug 04 18:58:57.648149 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.647806    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83517cba-548c-4784-9db9-6a1cf3a2c29a-istiod-ca-cert\") pod \"83517cba-548c-4784-9db9-6a1cf3a2c29a\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") "
Aug 04 18:58:57.648149 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.647832    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-socket\") pod \"83517cba-548c-4784-9db9-6a1cf3a2c29a\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") "
Aug 04 18:58:57.648149 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.647882    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-certs\") pod \"83517cba-548c-4784-9db9-6a1cf3a2c29a\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") "
Aug 04 18:58:57.648149 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.647912    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-token\") pod \"83517cba-548c-4784-9db9-6a1cf3a2c29a\" (UID: \"83517cba-548c-4784-9db9-6a1cf3a2c29a\") "
Aug 04 18:58:57.648149 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.647974    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83517cba-548c-4784-9db9-6a1cf3a2c29a" (UID: "83517cba-548c-4784-9db9-6a1cf3a2c29a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:58:57.648674 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.648172    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83517cba-548c-4784-9db9-6a1cf3a2c29a" (UID: "83517cba-548c-4784-9db9-6a1cf3a2c29a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:58:57.648674 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.648289    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83517cba-548c-4784-9db9-6a1cf3a2c29a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83517cba-548c-4784-9db9-6a1cf3a2c29a" (UID: "83517cba-548c-4784-9db9-6a1cf3a2c29a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:58:57.648674 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.648367    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83517cba-548c-4784-9db9-6a1cf3a2c29a" (UID: "83517cba-548c-4784-9db9-6a1cf3a2c29a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:58:57.648674 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.648425    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-credential-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:57.648674 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.648446    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:57.648674 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.648550    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-data" (OuterVolumeSpecName: "istio-data") pod "83517cba-548c-4784-9db9-6a1cf3a2c29a" (UID: "83517cba-548c-4784-9db9-6a1cf3a2c29a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:58:57.650098 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.650073    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-token" (OuterVolumeSpecName: "istio-token") pod "83517cba-548c-4784-9db9-6a1cf3a2c29a" (UID: "83517cba-548c-4784-9db9-6a1cf3a2c29a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:58:57.650229 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.650096    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83517cba-548c-4784-9db9-6a1cf3a2c29a" (UID: "83517cba-548c-4784-9db9-6a1cf3a2c29a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 18:58:57.650229 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.650107    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-kube-api-access-xxkzk" (OuterVolumeSpecName: "kube-api-access-xxkzk") pod "83517cba-548c-4784-9db9-6a1cf3a2c29a" (UID: "83517cba-548c-4784-9db9-6a1cf3a2c29a"). InnerVolumeSpecName "kube-api-access-xxkzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:58:57.650229 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.650212    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83517cba-548c-4784-9db9-6a1cf3a2c29a" (UID: "83517cba-548c-4784-9db9-6a1cf3a2c29a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:58:57.749300 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.749271    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-podinfo\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:57.749300 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.749295    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxkzk\" (UniqueName: \"kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-kube-api-access-xxkzk\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:57.749476 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.749309    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-envoy\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:57.749476 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.749321    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-data\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:57.749476 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.749332    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83517cba-548c-4784-9db9-6a1cf3a2c29a-istiod-ca-cert\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:57.749476 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.749341    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83517cba-548c-4784-9db9-6a1cf3a2c29a-workload-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:57.749476 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.749352    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83517cba-548c-4784-9db9-6a1cf3a2c29a-istio-token\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:58:57.895055 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.895024    2578 generic.go:356] "Generic (PLEG): container finished" podID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerID="add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c" exitCode=0
Aug 04 18:58:57.895225 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.895070    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"
Aug 04 18:58:57.895225 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.895095    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" event={"ID":"83517cba-548c-4784-9db9-6a1cf3a2c29a","Type":"ContainerDied","Data":"add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c"}
Aug 04 18:58:57.895225 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.895148    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc" event={"ID":"83517cba-548c-4784-9db9-6a1cf3a2c29a","Type":"ContainerDied","Data":"61934d05c3fc8e49c7ddd229c521dacba5d8efd95cd4c6025ed642f24f703fc1"}
Aug 04 18:58:57.895225 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.895171    2578 scope.go:117] "RemoveContainer" containerID="add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c"
Aug 04 18:58:57.904460 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.904445    2578 scope.go:117] "RemoveContainer" containerID="cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720"
Aug 04 18:58:57.912018 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.912004    2578 scope.go:117] "RemoveContainer" containerID="add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c"
Aug 04 18:58:57.912301 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:58:57.912273    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c\": container with ID starting with add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c not found: ID does not exist" containerID="add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c"
Aug 04 18:58:57.912373 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.912298    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c"} err="failed to get container status \"add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c\": rpc error: code = NotFound desc = could not find container \"add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c\": container with ID starting with add239a57e7bb3c1a65c958652c45d5fa65dcd25273dbfd295bca989a5f1b16c not found: ID does not exist"
Aug 04 18:58:57.912373 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.912319    2578 scope.go:117] "RemoveContainer" containerID="cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720"
Aug 04 18:58:57.912558 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:58:57.912540    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720\": container with ID starting with cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720 not found: ID does not exist" containerID="cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720"
Aug 04 18:58:57.912644 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.912561    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720"} err="failed to get container status \"cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720\": rpc error: code = NotFound desc = could not find container \"cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720\": container with ID starting with cd1eedfa85c4e08990e286a749e07d52d073ca17d6394fec00969ecce1d76720 not found: ID does not exist"
Aug 04 18:58:57.927977 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.927954    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"]
Aug 04 18:58:57.931529 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.931506    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-2646b40b-gw-openshift-default-9d68d8468-2dmcc"]
Aug 04 18:58:57.983530 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:57.983507    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:58.102160 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:58.102132    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:58:58.801661 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:58:58.801629    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" path="/var/lib/kubelet/pods/83517cba-548c-4784-9db9-6a1cf3a2c29a/volumes"
Aug 04 18:59:09.881363 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:09.881324    2578 patch_prober.go:28] interesting pod/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Aug 04 18:59:09.881782 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:09.881390    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Aug 04 18:59:09.963741 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:09.963706    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerID="68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2" exitCode=0
Aug 04 18:59:09.963895 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:09.963771    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" event={"ID":"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11","Type":"ContainerDied","Data":"68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2"}
Aug 04 18:59:09.964178 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:09.964160    2578 scope.go:117] "RemoveContainer" containerID="68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2"
Aug 04 18:59:09.965121 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:09.965091    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:59:09.965221 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:09.965160    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:59:09.965221 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:09.965189    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 18:59:10.970597 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:10.970543    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" event={"ID":"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11","Type":"ContainerStarted","Data":"f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4"}
Aug 04 18:59:11.315994 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:11.315960    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:59:11.316192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:11.316010    2578 patch_prober.go:28] interesting pod/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Aug 04 18:59:11.316192 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:11.316065    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Aug 04 18:59:12.314431 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:12.314394    2578 patch_prober.go:28] interesting pod/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Aug 04 18:59:12.315050 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:12.315016    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Aug 04 18:59:13.314445 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:13.314409    2578 patch_prober.go:28] interesting pod/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Aug 04 18:59:13.315026 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:13.314993    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Aug 04 18:59:14.314924 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:14.314889    2578 patch_prober.go:28] interesting pod/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Aug 04 18:59:14.315503 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:14.315473    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Aug 04 18:59:15.314853 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:15.314810    2578 patch_prober.go:28] interesting pod/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Aug 04 18:59:15.315059 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:15.314874    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Aug 04 18:59:16.318329 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:16.318299    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:59:16.318719 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:16.318540    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:59:16.319115 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:16.319101    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:59:27.066168 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:27.066121    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:27.166837 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:27.166807    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:27.270909 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:27.270881    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:27.366599 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:27.366481    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:28.486404 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:28.486374    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"]
Aug 04 18:59:28.486820 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:28.486623    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy" containerID="cri-o://f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4" gracePeriod=30
Aug 04 18:59:28.532647 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:28.532615    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:28.688180 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:28.688149    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:29.780180 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:29.780144    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:29.884660 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:29.884626    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:31.320819 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:31.320786    2578 patch_prober.go:28] interesting pod/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 18:59:31.321174 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:31.320846    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 18:59:31.375505 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:31.375474    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:31.468064 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:31.468036    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:32.511322 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:32.511289    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:33.090719 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.090687    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:33.185498 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.185462    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:33.253328 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.253292    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"]
Aug 04 18:59:33.253720 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.253703    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy"
Aug 04 18:59:33.253814 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.253722    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy"
Aug 04 18:59:33.253814 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.253739    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy"
Aug 04 18:59:33.253814 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.253747    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy"
Aug 04 18:59:33.253985 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.253836    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy"
Aug 04 18:59:33.253985 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.253849    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="83517cba-548c-4784-9db9-6a1cf3a2c29a" containerName="istio-proxy"
Aug 04 18:59:33.257337 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.257318    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 18:59:33.259934 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.259913    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-43a78dca\""
Aug 04 18:59:33.273130 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.273108    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"]
Aug 04 18:59:33.289657 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.289630    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 18:59:33.357473 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.357394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls87r\" (UniqueName: \"kubernetes.io/projected/d26ac579-0db0-42b6-89b5-ded91e07250f-kube-api-access-ls87r\") pod \"payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4\" (UID: \"d26ac579-0db0-42b6-89b5-ded91e07250f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 18:59:33.357473 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.357447    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d26ac579-0db0-42b6-89b5-ded91e07250f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4\" (UID: \"d26ac579-0db0-42b6-89b5-ded91e07250f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 18:59:33.458520 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.458490    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls87r\" (UniqueName: \"kubernetes.io/projected/d26ac579-0db0-42b6-89b5-ded91e07250f-kube-api-access-ls87r\") pod \"payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4\" (UID: \"d26ac579-0db0-42b6-89b5-ded91e07250f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 18:59:33.458678 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.458533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d26ac579-0db0-42b6-89b5-ded91e07250f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4\" (UID: \"d26ac579-0db0-42b6-89b5-ded91e07250f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 18:59:33.459227 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.459209    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d26ac579-0db0-42b6-89b5-ded91e07250f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4\" (UID: \"d26ac579-0db0-42b6-89b5-ded91e07250f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 18:59:33.470369 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.470346    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls87r\" (UniqueName: \"kubernetes.io/projected/d26ac579-0db0-42b6-89b5-ded91e07250f-kube-api-access-ls87r\") pod \"payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4\" (UID: \"d26ac579-0db0-42b6-89b5-ded91e07250f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 18:59:33.567539 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.567516    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 18:59:33.770504 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.770482    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:59:33.862953 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.862920    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-data\") pod \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") "
Aug 04 18:59:33.863110 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.862962    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-podinfo\") pod \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") "
Aug 04 18:59:33.863110 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863018    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-credential-socket\") pod \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") "
Aug 04 18:59:33.863110 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863040    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-certs\") pod \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") "
Aug 04 18:59:33.863110 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863079    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-envoy\") pod \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") "
Aug 04 18:59:33.863110 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863106    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istiod-ca-cert\") pod \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") "
Aug 04 18:59:33.863373 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863153    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" (UID: "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:59:33.863373 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863164    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-socket\") pod \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") "
Aug 04 18:59:33.863373 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863221    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-token\") pod \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") "
Aug 04 18:59:33.863373 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863273    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg6jm\" (UniqueName: \"kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-kube-api-access-hg6jm\") pod \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\" (UID: \"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11\") "
Aug 04 18:59:33.863373 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863348    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" (UID: "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:59:33.863875 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863358    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" (UID: "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:59:33.863875 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863476    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" (UID: "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:59:33.863875 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863501    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" (UID: "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 18:59:33.863875 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863631    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-credential-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:59:33.863875 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863667    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:59:33.863875 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863681    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istiod-ca-cert\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:59:33.863875 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863694    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-workload-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:59:33.863875 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.863707    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-data\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:59:33.865470 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.865448    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" (UID: "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:59:33.865564 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.865509    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-kube-api-access-hg6jm" (OuterVolumeSpecName: "kube-api-access-hg6jm") pod "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" (UID: "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11"). InnerVolumeSpecName "kube-api-access-hg6jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 18:59:33.865636 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.865587    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" (UID: "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 18:59:33.865636 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.865605    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" (UID: "9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 18:59:33.920983 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.920956    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"]
Aug 04 18:59:33.923274 ip-10-0-128-139 kubenswrapper[2578]: W0804 18:59:33.923252    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd26ac579_0db0_42b6_89b5_ded91e07250f.slice/crio-850c0f66d22d35cbc1ef909d36849497d3e50955f673746c966cd804a9b25f68 WatchSource:0}: Error finding container 850c0f66d22d35cbc1ef909d36849497d3e50955f673746c966cd804a9b25f68: Status 404 returned error can't find the container with id 850c0f66d22d35cbc1ef909d36849497d3e50955f673746c966cd804a9b25f68
Aug 04 18:59:33.964190 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.964172    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-podinfo\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:59:33.964190 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.964191    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-envoy\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:59:33.964318 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.964201    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-istio-token\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:59:33.964318 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:33.964209    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg6jm\" (UniqueName: \"kubernetes.io/projected/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11-kube-api-access-hg6jm\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 18:59:34.075359 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.075322    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerID="f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4" exitCode=0
Aug 04 18:59:34.075520 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.075373    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"
Aug 04 18:59:34.075520 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.075400    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" event={"ID":"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11","Type":"ContainerDied","Data":"f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4"}
Aug 04 18:59:34.075520 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.075451    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq" event={"ID":"9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11","Type":"ContainerDied","Data":"8c57524166b0f6abd04868e0322f069bdce2df1f37c348327d5ea276683bb830"}
Aug 04 18:59:34.075520 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.075477    2578 scope.go:117] "RemoveContainer" containerID="f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4"
Aug 04 18:59:34.077197 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.077174    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4" event={"ID":"d26ac579-0db0-42b6-89b5-ded91e07250f","Type":"ContainerStarted","Data":"aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5"}
Aug 04 18:59:34.077334 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.077210    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4" event={"ID":"d26ac579-0db0-42b6-89b5-ded91e07250f","Type":"ContainerStarted","Data":"850c0f66d22d35cbc1ef909d36849497d3e50955f673746c966cd804a9b25f68"}
Aug 04 18:59:34.077334 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.077300    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 18:59:34.086702 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.086682    2578 scope.go:117] "RemoveContainer" containerID="68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2"
Aug 04 18:59:34.094720 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.094703    2578 scope.go:117] "RemoveContainer" containerID="f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4"
Aug 04 18:59:34.094974 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:59:34.094956    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4\": container with ID starting with f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4 not found: ID does not exist" containerID="f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4"
Aug 04 18:59:34.095027 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.094982    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4"} err="failed to get container status \"f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4\": rpc error: code = NotFound desc = could not find container \"f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4\": container with ID starting with f13f81e439100ad414d1ad6075cff2bb5a951575879b9d90f599bc161c3ea4c4 not found: ID does not exist"
Aug 04 18:59:34.095027 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.094998    2578 scope.go:117] "RemoveContainer" containerID="68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2"
Aug 04 18:59:34.095245 ip-10-0-128-139 kubenswrapper[2578]: E0804 18:59:34.095227    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2\": container with ID starting with 68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2 not found: ID does not exist" containerID="68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2"
Aug 04 18:59:34.095301 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.095249    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2"} err="failed to get container status \"68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2\": rpc error: code = NotFound desc = could not find container \"68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2\": container with ID starting with 68b5936b1a3ae245e7917a17975f088e1b7c5ed893a78ab64ec567c8152133c2 not found: ID does not exist"
Aug 04 18:59:34.119403 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.119176    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4" podStartSLOduration=1.119160208 podStartE2EDuration="1.119160208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 18:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 18:59:34.118318413 +0000 UTC m=+1915.985233513" watchObservedRunningTime="2026-08-04 18:59:34.119160208 +0000 UTC m=+1915.986075312"
Aug 04 18:59:34.160528 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.160504    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"]
Aug 04 18:59:34.161945 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.161928    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-41669ddd-openshift-default-b9bd547c6-jrgdq"]
Aug 04 18:59:34.801571 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:34.801541    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" path="/var/lib/kubelet/pods/9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11/volumes"
Aug 04 18:59:45.084479 ip-10-0-128-139 kubenswrapper[2578]: I0804 18:59:45.084454    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 19:00:03.142666 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:03.142628    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"]
Aug 04 19:00:03.143148 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:03.142900    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4" podUID="d26ac579-0db0-42b6-89b5-ded91e07250f" containerName="payload-pre-processing" containerID="cri-o://aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5" gracePeriod=30
Aug 04 19:00:03.382171 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:03.382150    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 19:00:03.506744 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:03.506662    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ls87r\" (UniqueName: \"kubernetes.io/projected/d26ac579-0db0-42b6-89b5-ded91e07250f-kube-api-access-ls87r\") pod \"d26ac579-0db0-42b6-89b5-ded91e07250f\" (UID: \"d26ac579-0db0-42b6-89b5-ded91e07250f\") "
Aug 04 19:00:03.506744 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:03.506722    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d26ac579-0db0-42b6-89b5-ded91e07250f-plugins-config-volume\") pod \"d26ac579-0db0-42b6-89b5-ded91e07250f\" (UID: \"d26ac579-0db0-42b6-89b5-ded91e07250f\") "
Aug 04 19:00:03.507050 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:03.507024    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d26ac579-0db0-42b6-89b5-ded91e07250f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d26ac579-0db0-42b6-89b5-ded91e07250f" (UID: "d26ac579-0db0-42b6-89b5-ded91e07250f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:00:03.508663 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:03.508638    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d26ac579-0db0-42b6-89b5-ded91e07250f-kube-api-access-ls87r" (OuterVolumeSpecName: "kube-api-access-ls87r") pod "d26ac579-0db0-42b6-89b5-ded91e07250f" (UID: "d26ac579-0db0-42b6-89b5-ded91e07250f"). InnerVolumeSpecName "kube-api-access-ls87r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:00:03.608274 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:03.608242    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls87r\" (UniqueName: \"kubernetes.io/projected/d26ac579-0db0-42b6-89b5-ded91e07250f-kube-api-access-ls87r\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:00:03.608274 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:03.608268    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d26ac579-0db0-42b6-89b5-ded91e07250f-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:00:03.701159 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:03.701126    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:03.856555 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:03.856526    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:04.215179 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.215097    2578 generic.go:356] "Generic (PLEG): container finished" podID="d26ac579-0db0-42b6-89b5-ded91e07250f" containerID="aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5" exitCode=0
Aug 04 19:00:04.215179 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.215148    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"
Aug 04 19:00:04.215179 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.215173    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4" event={"ID":"d26ac579-0db0-42b6-89b5-ded91e07250f","Type":"ContainerDied","Data":"aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5"}
Aug 04 19:00:04.215713 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.215224    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4" event={"ID":"d26ac579-0db0-42b6-89b5-ded91e07250f","Type":"ContainerDied","Data":"850c0f66d22d35cbc1ef909d36849497d3e50955f673746c966cd804a9b25f68"}
Aug 04 19:00:04.215713 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.215254    2578 scope.go:117] "RemoveContainer" containerID="aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5"
Aug 04 19:00:04.224194 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.224176    2578 scope.go:117] "RemoveContainer" containerID="aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5"
Aug 04 19:00:04.224460 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:00:04.224440    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5\": container with ID starting with aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5 not found: ID does not exist" containerID="aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5"
Aug 04 19:00:04.224512 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.224468    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5"} err="failed to get container status \"aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5\": rpc error: code = NotFound desc = could not find container \"aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5\": container with ID starting with aea7743d9dd4aaab9ec49edb3dc471878f73205a25702f391b2af0500a7a9ad5 not found: ID does not exist"
Aug 04 19:00:04.244894 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.244870    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"]
Aug 04 19:00:04.268469 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.268444    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-43a78dca-7c46db4744-8kmc4"]
Aug 04 19:00:04.433417 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.433383    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:04.522275 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.522203    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:04.801482 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:04.801452    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d26ac579-0db0-42b6-89b5-ded91e07250f" path="/var/lib/kubelet/pods/d26ac579-0db0-42b6-89b5-ded91e07250f/volumes"
Aug 04 19:00:05.533641 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:05.533609    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:15.991122 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:15.991084    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:23.633618 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:23.633567    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:23.746780 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:23.746750    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:23.846114 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:23.846081    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:23.944542 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:23.944468    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:25.160912 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:25.160880    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:36.897976 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:36.897895    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:50.310120 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:50.310083    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:57.227703 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:57.227666    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:57.440600 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:57.440539    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:00:57.577913 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:00:57.577880    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:13.309535 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:13.309500    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:19.579130 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:19.579095    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:19.682634 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:19.682567    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:19.781689 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:19.781658    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:30.267505 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:30.267466    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:41.266500 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:41.266464    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:48.104106 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:48.104073    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:48.259166 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:48.259131    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:48.361103 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:48.361026    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:49.127967 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:49.127940    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:49.222969 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:49.222943    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:50.864681 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:50.864645    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:59.051623 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:59.051592    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:59.156015 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:59.155979    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:01:59.738139 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:01:59.738103    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:06.980623 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:06.980525    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:07.115997 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:07.115959    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:09.044751 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:09.044715    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:09.143499 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:09.143471    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:10.845626 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:10.845570    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:10.942495 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:10.942460    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:12.071532 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.071499    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:12.559988 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.559955    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:12.666862 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.666832    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:12.675871 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.675844    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"]
Aug 04 19:02:12.676199 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.676186    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy"
Aug 04 19:02:12.676248 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.676201    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy"
Aug 04 19:02:12.676248 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.676215    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d26ac579-0db0-42b6-89b5-ded91e07250f" containerName="payload-pre-processing"
Aug 04 19:02:12.676248 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.676220    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d26ac579-0db0-42b6-89b5-ded91e07250f" containerName="payload-pre-processing"
Aug 04 19:02:12.676248 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.676233    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy"
Aug 04 19:02:12.676248 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.676239    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy"
Aug 04 19:02:12.676395 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.676290    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d26ac579-0db0-42b6-89b5-ded91e07250f" containerName="payload-pre-processing"
Aug 04 19:02:12.676395 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.676299    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy"
Aug 04 19:02:12.676395 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.676305    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ca5f0a0-cfb9-4d25-949e-8cb4957e9e11" containerName="istio-proxy"
Aug 04 19:02:12.679233 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.679217    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:12.682078 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.682060    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2f0c212d-dockercfg-zhw72\""
Aug 04 19:02:12.682195 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.682064    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2f0c212d\""
Aug 04 19:02:12.686131 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.685823    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"]
Aug 04 19:02:12.689186 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.689168    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:12.691062 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.691037    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"]
Aug 04 19:02:12.702766 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.702745    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"]
Aug 04 19:02:12.795320 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.795277    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grjvs\" (UniqueName: \"kubernetes.io/projected/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-kube-api-access-grjvs\") pod \"payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq\" (UID: \"cdd48eb2-6238-43d3-b27e-0d84f0126dbd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:12.795514 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.795327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e47f257-e87a-4e60-8f66-3562f8211440-plugins-config-volume\") pod \"payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54\" (UID: \"8e47f257-e87a-4e60-8f66-3562f8211440\") " pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:12.795514 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.795395    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq\" (UID: \"cdd48eb2-6238-43d3-b27e-0d84f0126dbd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:12.795514 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.795424    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dts9h\" (UniqueName: \"kubernetes.io/projected/8e47f257-e87a-4e60-8f66-3562f8211440-kube-api-access-dts9h\") pod \"payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54\" (UID: \"8e47f257-e87a-4e60-8f66-3562f8211440\") " pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:12.896181 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.896078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e47f257-e87a-4e60-8f66-3562f8211440-plugins-config-volume\") pod \"payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54\" (UID: \"8e47f257-e87a-4e60-8f66-3562f8211440\") " pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:12.896181 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.896126    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq\" (UID: \"cdd48eb2-6238-43d3-b27e-0d84f0126dbd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:12.896416 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.896248    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dts9h\" (UniqueName: \"kubernetes.io/projected/8e47f257-e87a-4e60-8f66-3562f8211440-kube-api-access-dts9h\") pod \"payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54\" (UID: \"8e47f257-e87a-4e60-8f66-3562f8211440\") " pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:12.896475 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.896425    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grjvs\" (UniqueName: \"kubernetes.io/projected/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-kube-api-access-grjvs\") pod \"payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq\" (UID: \"cdd48eb2-6238-43d3-b27e-0d84f0126dbd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:12.896942 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.896921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq\" (UID: \"cdd48eb2-6238-43d3-b27e-0d84f0126dbd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:12.897035 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.896970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e47f257-e87a-4e60-8f66-3562f8211440-plugins-config-volume\") pod \"payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54\" (UID: \"8e47f257-e87a-4e60-8f66-3562f8211440\") " pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:12.909665 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.909635    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dts9h\" (UniqueName: \"kubernetes.io/projected/8e47f257-e87a-4e60-8f66-3562f8211440-kube-api-access-dts9h\") pod \"payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54\" (UID: \"8e47f257-e87a-4e60-8f66-3562f8211440\") " pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:12.909765 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.909686    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grjvs\" (UniqueName: \"kubernetes.io/projected/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-kube-api-access-grjvs\") pod \"payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq\" (UID: \"cdd48eb2-6238-43d3-b27e-0d84f0126dbd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:12.990121 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:12.990085    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:13.001943 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:13.001914    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:13.142623 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:13.142594    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"]
Aug 04 19:02:13.143980 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:02:13.143952    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdd48eb2_6238_43d3_b27e_0d84f0126dbd.slice/crio-ebdc43c01223a8da6f05f3fc0c0232d2a772546551ef347bd8b95f0d0cc017d2 WatchSource:0}: Error finding container ebdc43c01223a8da6f05f3fc0c0232d2a772546551ef347bd8b95f0d0cc017d2: Status 404 returned error can't find the container with id ebdc43c01223a8da6f05f3fc0c0232d2a772546551ef347bd8b95f0d0cc017d2
Aug 04 19:02:13.161042 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:13.161011    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"]
Aug 04 19:02:13.162983 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:02:13.162958    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e47f257_e87a_4e60_8f66_3562f8211440.slice/crio-70f29578e1ca042423881f79fca5ea179fd82ff1b16cfc6c53fd90da2b7deeb1 WatchSource:0}: Error finding container 70f29578e1ca042423881f79fca5ea179fd82ff1b16cfc6c53fd90da2b7deeb1: Status 404 returned error can't find the container with id 70f29578e1ca042423881f79fca5ea179fd82ff1b16cfc6c53fd90da2b7deeb1
Aug 04 19:02:13.807698 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:13.807660    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq" event={"ID":"cdd48eb2-6238-43d3-b27e-0d84f0126dbd","Type":"ContainerStarted","Data":"fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e"}
Aug 04 19:02:13.807886 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:13.807715    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq" event={"ID":"cdd48eb2-6238-43d3-b27e-0d84f0126dbd","Type":"ContainerStarted","Data":"ebdc43c01223a8da6f05f3fc0c0232d2a772546551ef347bd8b95f0d0cc017d2"}
Aug 04 19:02:13.807886 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:13.807740    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:13.809188 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:13.809164    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54" event={"ID":"8e47f257-e87a-4e60-8f66-3562f8211440","Type":"ContainerStarted","Data":"1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786"}
Aug 04 19:02:13.809320 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:13.809206    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54" event={"ID":"8e47f257-e87a-4e60-8f66-3562f8211440","Type":"ContainerStarted","Data":"70f29578e1ca042423881f79fca5ea179fd82ff1b16cfc6c53fd90da2b7deeb1"}
Aug 04 19:02:13.809320 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:13.809230    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:13.828952 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:13.828909    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq" podStartSLOduration=1.828894148 podStartE2EDuration="1.828894148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:02:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:02:13.828276009 +0000 UTC m=+2075.695191109" watchObservedRunningTime="2026-08-04 19:02:13.828894148 +0000 UTC m=+2075.695809247"
Aug 04 19:02:13.852847 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:13.852796    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54" podStartSLOduration=1.85278127 podStartE2EDuration="1.85278127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:02:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:02:13.851395748 +0000 UTC m=+2075.718310847" watchObservedRunningTime="2026-08-04 19:02:13.85278127 +0000 UTC m=+2075.719696369"
Aug 04 19:02:24.817279 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:24.817233    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq" podUID="cdd48eb2-6238-43d3-b27e-0d84f0126dbd" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.42:9004: connect: connection refused"
Aug 04 19:02:24.817802 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:24.817503    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:34.816443 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:34.816412    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:35.851550 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:35.851514    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:36.594924 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:36.594877    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:36.705145 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:36.705112    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:38.218546 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:38.218514    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:43.552717 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:43.552684    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:43.650200 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:43.650170    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:54.516539 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.516502    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:54.597624 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.597595    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"]
Aug 04 19:02:54.597841 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.597817    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54" podUID="8e47f257-e87a-4e60-8f66-3562f8211440" containerName="payload-processing" containerID="cri-o://1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786" gracePeriod=30
Aug 04 19:02:54.622790 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.622765    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"]
Aug 04 19:02:54.623017 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.622995    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq" podUID="cdd48eb2-6238-43d3-b27e-0d84f0126dbd" containerName="payload-pre-processing" containerID="cri-o://fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e" gracePeriod=30
Aug 04 19:02:54.816119 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.816078    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq" podUID="cdd48eb2-6238-43d3-b27e-0d84f0126dbd" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.42:9004: connect: connection refused"
Aug 04 19:02:54.878124 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.878098    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:54.903343 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.903321    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:54.963315 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.963292    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grjvs\" (UniqueName: \"kubernetes.io/projected/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-kube-api-access-grjvs\") pod \"cdd48eb2-6238-43d3-b27e-0d84f0126dbd\" (UID: \"cdd48eb2-6238-43d3-b27e-0d84f0126dbd\") "
Aug 04 19:02:54.963482 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.963347    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e47f257-e87a-4e60-8f66-3562f8211440-plugins-config-volume\") pod \"8e47f257-e87a-4e60-8f66-3562f8211440\" (UID: \"8e47f257-e87a-4e60-8f66-3562f8211440\") "
Aug 04 19:02:54.963482 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.963467    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-plugins-config-volume\") pod \"cdd48eb2-6238-43d3-b27e-0d84f0126dbd\" (UID: \"cdd48eb2-6238-43d3-b27e-0d84f0126dbd\") "
Aug 04 19:02:54.963652 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.963545    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dts9h\" (UniqueName: \"kubernetes.io/projected/8e47f257-e87a-4e60-8f66-3562f8211440-kube-api-access-dts9h\") pod \"8e47f257-e87a-4e60-8f66-3562f8211440\" (UID: \"8e47f257-e87a-4e60-8f66-3562f8211440\") "
Aug 04 19:02:54.963783 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.963762    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e47f257-e87a-4e60-8f66-3562f8211440-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e47f257-e87a-4e60-8f66-3562f8211440" (UID: "8e47f257-e87a-4e60-8f66-3562f8211440"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:02:54.963833 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.963796    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cdd48eb2-6238-43d3-b27e-0d84f0126dbd" (UID: "cdd48eb2-6238-43d3-b27e-0d84f0126dbd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:02:54.963917 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.963900    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e47f257-e87a-4e60-8f66-3562f8211440-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:02:54.963953 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.963924    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:02:54.965387 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.965367    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-kube-api-access-grjvs" (OuterVolumeSpecName: "kube-api-access-grjvs") pod "cdd48eb2-6238-43d3-b27e-0d84f0126dbd" (UID: "cdd48eb2-6238-43d3-b27e-0d84f0126dbd"). InnerVolumeSpecName "kube-api-access-grjvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:02:54.965460 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.965389    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e47f257-e87a-4e60-8f66-3562f8211440-kube-api-access-dts9h" (OuterVolumeSpecName: "kube-api-access-dts9h") pod "8e47f257-e87a-4e60-8f66-3562f8211440" (UID: "8e47f257-e87a-4e60-8f66-3562f8211440"). InnerVolumeSpecName "kube-api-access-dts9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:02:54.998873 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.998848    2578 generic.go:356] "Generic (PLEG): container finished" podID="cdd48eb2-6238-43d3-b27e-0d84f0126dbd" containerID="fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e" exitCode=0
Aug 04 19:02:54.998973 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.998887    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"
Aug 04 19:02:54.998973 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.998929    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq" event={"ID":"cdd48eb2-6238-43d3-b27e-0d84f0126dbd","Type":"ContainerDied","Data":"fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e"}
Aug 04 19:02:54.999062 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.998974    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq" event={"ID":"cdd48eb2-6238-43d3-b27e-0d84f0126dbd","Type":"ContainerDied","Data":"ebdc43c01223a8da6f05f3fc0c0232d2a772546551ef347bd8b95f0d0cc017d2"}
Aug 04 19:02:54.999062 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:54.998997    2578 scope.go:117] "RemoveContainer" containerID="fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e"
Aug 04 19:02:55.000083 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.000063    2578 generic.go:356] "Generic (PLEG): container finished" podID="8e47f257-e87a-4e60-8f66-3562f8211440" containerID="1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786" exitCode=0
Aug 04 19:02:55.000165 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.000095    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"
Aug 04 19:02:55.000165 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.000142    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54" event={"ID":"8e47f257-e87a-4e60-8f66-3562f8211440","Type":"ContainerDied","Data":"1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786"}
Aug 04 19:02:55.000233 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.000192    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54" event={"ID":"8e47f257-e87a-4e60-8f66-3562f8211440","Type":"ContainerDied","Data":"70f29578e1ca042423881f79fca5ea179fd82ff1b16cfc6c53fd90da2b7deeb1"}
Aug 04 19:02:55.007156 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.007138    2578 scope.go:117] "RemoveContainer" containerID="fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e"
Aug 04 19:02:55.007439 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:02:55.007422    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e\": container with ID starting with fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e not found: ID does not exist" containerID="fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e"
Aug 04 19:02:55.007502 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.007444    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e"} err="failed to get container status \"fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e\": rpc error: code = NotFound desc = could not find container \"fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e\": container with ID starting with fc27f8308d6182b34841e30f182c55720b570f0b5ef7629319a7bc339821827e not found: ID does not exist"
Aug 04 19:02:55.007502 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.007459    2578 scope.go:117] "RemoveContainer" containerID="1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786"
Aug 04 19:02:55.014942 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.014925    2578 scope.go:117] "RemoveContainer" containerID="1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786"
Aug 04 19:02:55.015190 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:02:55.015170    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786\": container with ID starting with 1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786 not found: ID does not exist" containerID="1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786"
Aug 04 19:02:55.015241 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.015201    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786"} err="failed to get container status \"1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786\": rpc error: code = NotFound desc = could not find container \"1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786\": container with ID starting with 1a3faa2238c5713a2ae0ac852374eeedddcd567717bd964403637d049b0c3786 not found: ID does not exist"
Aug 04 19:02:55.027067 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.027041    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"]
Aug 04 19:02:55.038114 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.038097    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54"]
Aug 04 19:02:55.049525 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.049507    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"]
Aug 04 19:02:55.059721 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.059699    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2f0c212d-7559fd6ccb-96ntq"]
Aug 04 19:02:55.064559 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.064541    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dts9h\" (UniqueName: \"kubernetes.io/projected/8e47f257-e87a-4e60-8f66-3562f8211440-kube-api-access-dts9h\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:02:55.064631 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.064561    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grjvs\" (UniqueName: \"kubernetes.io/projected/cdd48eb2-6238-43d3-b27e-0d84f0126dbd-kube-api-access-grjvs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:02:55.816637 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:55.816563    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-2f0c212d-847c87d778-x5h54" podUID="8e47f257-e87a-4e60-8f66-3562f8211440" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.43:9004: i/o timeout"
Aug 04 19:02:56.502373 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:56.502332    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:02:56.801505 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:56.801472    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e47f257-e87a-4e60-8f66-3562f8211440" path="/var/lib/kubelet/pods/8e47f257-e87a-4e60-8f66-3562f8211440/volumes"
Aug 04 19:02:56.801854 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:02:56.801841    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdd48eb2-6238-43d3-b27e-0d84f0126dbd" path="/var/lib/kubelet/pods/cdd48eb2-6238-43d3-b27e-0d84f0126dbd/volumes"
Aug 04 19:03:01.572781 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:01.572747    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:01.645444 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:01.645411    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:01.885858 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:01.885785    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:02.062074 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:02.062043    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:02.205234 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:02.205155    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:02.333733 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:02.333701    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:02.427496 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:02.427469    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:09.409453 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:09.409419    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:09.519712 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:09.519680    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:09.618132 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:09.618102    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:09.799827 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:09.799798    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:15.980706 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:15.980671    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:16.084411 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:16.084377    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:03:44.276338 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:44.276304    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"]
Aug 04 19:03:44.276818 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:44.276507    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg" podUID="d7d0caee-a593-42c1-9dd9-3723b03244ed" containerName="manager" containerID="cri-o://4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03" gracePeriod=10
Aug 04 19:03:44.330977 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:03:44.330944    2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d0caee_a593_42c1_9dd9_3723b03244ed.slice/crio-conmon-4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03.scope\": RecentStats: unable to find data in memory cache]"
Aug 04 19:03:44.525110 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:44.525088    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 19:03:44.602468 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:44.602433    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d0caee-a593-42c1-9dd9-3723b03244ed-extensions-socket-volume\") pod \"d7d0caee-a593-42c1-9dd9-3723b03244ed\" (UID: \"d7d0caee-a593-42c1-9dd9-3723b03244ed\") "
Aug 04 19:03:44.602654 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:44.602515    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prsvg\" (UniqueName: \"kubernetes.io/projected/d7d0caee-a593-42c1-9dd9-3723b03244ed-kube-api-access-prsvg\") pod \"d7d0caee-a593-42c1-9dd9-3723b03244ed\" (UID: \"d7d0caee-a593-42c1-9dd9-3723b03244ed\") "
Aug 04 19:03:44.602851 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:44.602823    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d0caee-a593-42c1-9dd9-3723b03244ed-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d7d0caee-a593-42c1-9dd9-3723b03244ed" (UID: "d7d0caee-a593-42c1-9dd9-3723b03244ed"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:03:44.604450 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:44.604429    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d0caee-a593-42c1-9dd9-3723b03244ed-kube-api-access-prsvg" (OuterVolumeSpecName: "kube-api-access-prsvg") pod "d7d0caee-a593-42c1-9dd9-3723b03244ed" (UID: "d7d0caee-a593-42c1-9dd9-3723b03244ed"). InnerVolumeSpecName "kube-api-access-prsvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:03:44.703162 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:44.703124    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d0caee-a593-42c1-9dd9-3723b03244ed-extensions-socket-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:03:44.703162 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:44.703155    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prsvg\" (UniqueName: \"kubernetes.io/projected/d7d0caee-a593-42c1-9dd9-3723b03244ed-kube-api-access-prsvg\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:03:45.229520 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:45.229489    2578 generic.go:356] "Generic (PLEG): container finished" podID="d7d0caee-a593-42c1-9dd9-3723b03244ed" containerID="4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03" exitCode=0
Aug 04 19:03:45.229732 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:45.229532    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"
Aug 04 19:03:45.229732 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:45.229600    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg" event={"ID":"d7d0caee-a593-42c1-9dd9-3723b03244ed","Type":"ContainerDied","Data":"4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03"}
Aug 04 19:03:45.229732 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:45.229652    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg" event={"ID":"d7d0caee-a593-42c1-9dd9-3723b03244ed","Type":"ContainerDied","Data":"08f11cfc96916903c6b136d14236e35ae0ce69f6487c8caa05ad5e35b0a523bf"}
Aug 04 19:03:45.229732 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:45.229673    2578 scope.go:117] "RemoveContainer" containerID="4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03"
Aug 04 19:03:45.238923 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:45.238904    2578 scope.go:117] "RemoveContainer" containerID="4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03"
Aug 04 19:03:45.239193 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:03:45.239171    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03\": container with ID starting with 4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03 not found: ID does not exist" containerID="4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03"
Aug 04 19:03:45.239282 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:45.239199    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03"} err="failed to get container status \"4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03\": rpc error: code = NotFound desc = could not find container \"4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03\": container with ID starting with 4631a4c6e734a21dbfc216a653f74987b51828b86431f86a09d9d07b17768c03 not found: ID does not exist"
Aug 04 19:03:45.250237 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:45.250203    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"]
Aug 04 19:03:45.257207 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:45.257183    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-g4xpg"]
Aug 04 19:03:46.801634 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:03:46.801597    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7d0caee-a593-42c1-9dd9-3723b03244ed" path="/var/lib/kubelet/pods/d7d0caee-a593-42c1-9dd9-3723b03244ed/volumes"
Aug 04 19:04:49.257271 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:49.257238    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:04:59.918634 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.918599    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"]
Aug 04 19:04:59.919035 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.918913    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7d0caee-a593-42c1-9dd9-3723b03244ed" containerName="manager"
Aug 04 19:04:59.919035 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.918923    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d0caee-a593-42c1-9dd9-3723b03244ed" containerName="manager"
Aug 04 19:04:59.919035 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.918933    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd48eb2-6238-43d3-b27e-0d84f0126dbd" containerName="payload-pre-processing"
Aug 04 19:04:59.919035 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.918939    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd48eb2-6238-43d3-b27e-0d84f0126dbd" containerName="payload-pre-processing"
Aug 04 19:04:59.919035 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.918954    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e47f257-e87a-4e60-8f66-3562f8211440" containerName="payload-processing"
Aug 04 19:04:59.919035 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.918960    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e47f257-e87a-4e60-8f66-3562f8211440" containerName="payload-processing"
Aug 04 19:04:59.919035 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.919011    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7d0caee-a593-42c1-9dd9-3723b03244ed" containerName="manager"
Aug 04 19:04:59.919035 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.919019    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e47f257-e87a-4e60-8f66-3562f8211440" containerName="payload-processing"
Aug 04 19:04:59.919035 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.919024    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd48eb2-6238-43d3-b27e-0d84f0126dbd" containerName="payload-pre-processing"
Aug 04 19:04:59.921916 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.921899    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"
Aug 04 19:04:59.925334 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.925316    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j8gx9\""
Aug 04 19:04:59.943951 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:04:59.943923    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"]
Aug 04 19:05:00.042171 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.042134    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jzls\" (UniqueName: \"kubernetes.io/projected/2b1a32d3-57c0-4a09-9ee1-733b0350be45-kube-api-access-6jzls\") pod \"kuadrant-operator-controller-manager-68cf589589-2v4vs\" (UID: \"2b1a32d3-57c0-4a09-9ee1-733b0350be45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"
Aug 04 19:05:00.042357 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.042199    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b1a32d3-57c0-4a09-9ee1-733b0350be45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-2v4vs\" (UID: \"2b1a32d3-57c0-4a09-9ee1-733b0350be45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"
Aug 04 19:05:00.143497 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.143459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b1a32d3-57c0-4a09-9ee1-733b0350be45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-2v4vs\" (UID: \"2b1a32d3-57c0-4a09-9ee1-733b0350be45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"
Aug 04 19:05:00.143706 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.143524    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jzls\" (UniqueName: \"kubernetes.io/projected/2b1a32d3-57c0-4a09-9ee1-733b0350be45-kube-api-access-6jzls\") pod \"kuadrant-operator-controller-manager-68cf589589-2v4vs\" (UID: \"2b1a32d3-57c0-4a09-9ee1-733b0350be45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"
Aug 04 19:05:00.143905 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.143884    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b1a32d3-57c0-4a09-9ee1-733b0350be45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-2v4vs\" (UID: \"2b1a32d3-57c0-4a09-9ee1-733b0350be45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"
Aug 04 19:05:00.153778 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.153758    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jzls\" (UniqueName: \"kubernetes.io/projected/2b1a32d3-57c0-4a09-9ee1-733b0350be45-kube-api-access-6jzls\") pod \"kuadrant-operator-controller-manager-68cf589589-2v4vs\" (UID: \"2b1a32d3-57c0-4a09-9ee1-733b0350be45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"
Aug 04 19:05:00.231964 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.231875    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"
Aug 04 19:05:00.377936 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.377876    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"]
Aug 04 19:05:00.382303 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:05:00.382276    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1a32d3_57c0_4a09_9ee1_733b0350be45.slice/crio-0c9397afa010453d4bbeb7db782e43fcc0eea43ec18ae7c22104c84b6493fb21 WatchSource:0}: Error finding container 0c9397afa010453d4bbeb7db782e43fcc0eea43ec18ae7c22104c84b6493fb21: Status 404 returned error can't find the container with id 0c9397afa010453d4bbeb7db782e43fcc0eea43ec18ae7c22104c84b6493fb21
Aug 04 19:05:00.551879 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.551841    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs" event={"ID":"2b1a32d3-57c0-4a09-9ee1-733b0350be45","Type":"ContainerStarted","Data":"e33d5c5a473742737d58d1faf4c0633c4aa4e5c11ccfae156d060cdf582981aa"}
Aug 04 19:05:00.552055 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.551905    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs" event={"ID":"2b1a32d3-57c0-4a09-9ee1-733b0350be45","Type":"ContainerStarted","Data":"0c9397afa010453d4bbeb7db782e43fcc0eea43ec18ae7c22104c84b6493fb21"}
Aug 04 19:05:00.552055 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.551987    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"
Aug 04 19:05:00.578873 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:00.578824    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs" podStartSLOduration=1.5788062269999998 podStartE2EDuration="1.578806227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:04:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:05:00.577670901 +0000 UTC m=+2242.444586004" watchObservedRunningTime="2026-08-04 19:05:00.578806227 +0000 UTC m=+2242.445721328"
Aug 04 19:05:11.559457 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:11.559422    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-2v4vs"
Aug 04 19:05:18.043371 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.043341    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:05:18.408803 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.408725    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"]
Aug 04 19:05:18.412014 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.411986    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.418037 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.418016    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-59f3c9c8-openshift-default-dockercfg-rrg4l\""
Aug 04 19:05:18.455865 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.455839    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"]
Aug 04 19:05:18.491230 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.491195    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:05:18.496267 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.496240    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-credential-socket\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.496394 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.496274    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-certs\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.496394 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.496306    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-socket\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.496394 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.496378    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-data\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.496530 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.496408    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-envoy\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.496530 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.496437    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-token\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.496530 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.496461    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istiod-ca-cert\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.496530 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.496487    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hngxn\" (UniqueName: \"kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-kube-api-access-hngxn\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.496696 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.496542    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-podinfo\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597117 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-data\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597117 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597124    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-envoy\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597298 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597141    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-token\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597298 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597174    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istiod-ca-cert\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597298 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597210    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hngxn\" (UniqueName: \"kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-kube-api-access-hngxn\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597298 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-podinfo\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597520 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-credential-socket\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597520 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-certs\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597520 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597353    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-socket\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597520 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597476    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-data\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597773 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597663    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-socket\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597835 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-certs\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597897 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597840    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-credential-socket\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.597941 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.597927    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istiod-ca-cert\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.599437 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.599412    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-envoy\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.599652 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.599635    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-podinfo\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.605510 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.605491    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hngxn\" (UniqueName: \"kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-kube-api-access-hngxn\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.606164 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.606142    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-token\") pod \"e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") " pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.725520 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.725440    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:18.861668 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.861643    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"]
Aug 04 19:05:18.863412 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:05:18.863382    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f21e5d5_8611_439f_9c86_1f5162f8ab55.slice/crio-081516a1f31310f476d6e9c087e499ffaf7e4c61b7b2febfc34d66a47ba34596 WatchSource:0}: Error finding container 081516a1f31310f476d6e9c087e499ffaf7e4c61b7b2febfc34d66a47ba34596: Status 404 returned error can't find the container with id 081516a1f31310f476d6e9c087e499ffaf7e4c61b7b2febfc34d66a47ba34596
Aug 04 19:05:18.865629 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.865564    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:05:18.865712 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.865673    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:05:18.865754 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:18.865718    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:05:19.642457 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:19.642423    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" event={"ID":"5f21e5d5-8611-439f-9c86-1f5162f8ab55","Type":"ContainerStarted","Data":"0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342"}
Aug 04 19:05:19.642889 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:19.642475    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" event={"ID":"5f21e5d5-8611-439f-9c86-1f5162f8ab55","Type":"ContainerStarted","Data":"081516a1f31310f476d6e9c087e499ffaf7e4c61b7b2febfc34d66a47ba34596"}
Aug 04 19:05:19.666393 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:19.666351    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" podStartSLOduration=1.666334842 podStartE2EDuration="1.666334842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:05:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:05:19.663856392 +0000 UTC m=+2261.530771493" watchObservedRunningTime="2026-08-04 19:05:19.666334842 +0000 UTC m=+2261.533249943"
Aug 04 19:05:19.726012 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:19.725984    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:19.731312 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:19.731293    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:20.256723 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:20.256690    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:05:20.647513 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:20.647481    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:20.648452 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:20.648427    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:21.588696 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:21.588649    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:05:21.672105 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:21.672059    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:05:22.711431 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:22.711385    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:05:22.813900 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:22.813860    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:05:23.371461 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.371431    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:05:23.472502 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.472468    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:05:23.514243 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.514211    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"]
Aug 04 19:05:23.518162 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.518147    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:05:23.522387 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.522369    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-59f3c9c8\""
Aug 04 19:05:23.539614 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.539590    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"]
Aug 04 19:05:23.541028 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.541002    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/876f5c85-b5db-4626-a267-5aad04146561-plugins-config-volume\") pod \"payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7\" (UID: \"876f5c85-b5db-4626-a267-5aad04146561\") " pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:05:23.541167 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.541152    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws8bn\" (UniqueName: \"kubernetes.io/projected/876f5c85-b5db-4626-a267-5aad04146561-kube-api-access-ws8bn\") pod \"payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7\" (UID: \"876f5c85-b5db-4626-a267-5aad04146561\") " pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:05:23.557862 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.557835    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"]
Aug 04 19:05:23.560993 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.560974    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:05:23.564611 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.564590    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-59f3c9c8-dockercfg-qczzs\""
Aug 04 19:05:23.588939 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.588918    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"]
Aug 04 19:05:23.642159 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.642061    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws8bn\" (UniqueName: \"kubernetes.io/projected/876f5c85-b5db-4626-a267-5aad04146561-kube-api-access-ws8bn\") pod \"payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7\" (UID: \"876f5c85-b5db-4626-a267-5aad04146561\") " pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:05:23.642159 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.642132    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/876f5c85-b5db-4626-a267-5aad04146561-plugins-config-volume\") pod \"payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7\" (UID: \"876f5c85-b5db-4626-a267-5aad04146561\") " pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:05:23.642382 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.642241    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7\" (UID: \"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:05:23.642382 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.642279    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxbf8\" (UniqueName: \"kubernetes.io/projected/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-kube-api-access-gxbf8\") pod \"payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7\" (UID: \"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:05:23.643341 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.643310    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/876f5c85-b5db-4626-a267-5aad04146561-plugins-config-volume\") pod \"payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7\" (UID: \"876f5c85-b5db-4626-a267-5aad04146561\") " pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:05:23.665935 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.665903    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws8bn\" (UniqueName: \"kubernetes.io/projected/876f5c85-b5db-4626-a267-5aad04146561-kube-api-access-ws8bn\") pod \"payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7\" (UID: \"876f5c85-b5db-4626-a267-5aad04146561\") " pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:05:23.743153 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.743112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7\" (UID: \"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:05:23.743517 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.743159    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxbf8\" (UniqueName: \"kubernetes.io/projected/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-kube-api-access-gxbf8\") pod \"payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7\" (UID: \"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:05:23.743906 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.743888    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7\" (UID: \"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:05:23.777557 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.777531    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxbf8\" (UniqueName: \"kubernetes.io/projected/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-kube-api-access-gxbf8\") pod \"payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7\" (UID: \"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:05:23.828427 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.828402    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:05:23.870871 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.870840    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:05:23.977796 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:23.977767    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"]
Aug 04 19:05:23.980059 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:05:23.980023    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod876f5c85_b5db_4626_a267_5aad04146561.slice/crio-93768b21a36ff2f178a18652cae9646b1cf6ee21b40395ac05c938cbc83ac4af WatchSource:0}: Error finding container 93768b21a36ff2f178a18652cae9646b1cf6ee21b40395ac05c938cbc83ac4af: Status 404 returned error can't find the container with id 93768b21a36ff2f178a18652cae9646b1cf6ee21b40395ac05c938cbc83ac4af
Aug 04 19:05:24.027205 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:24.027183    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"]
Aug 04 19:05:24.029076 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:05:24.029052    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00d9ba8_6a03_47ad_9bd8_8b275dcc6cac.slice/crio-c3728b1529071f580ac3a387d336fb9990e19d5660b5999e5137f97181460073 WatchSource:0}: Error finding container c3728b1529071f580ac3a387d336fb9990e19d5660b5999e5137f97181460073: Status 404 returned error can't find the container with id c3728b1529071f580ac3a387d336fb9990e19d5660b5999e5137f97181460073
Aug 04 19:05:24.667425 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:24.667389    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7" event={"ID":"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac","Type":"ContainerStarted","Data":"401fcc06fc6f4279b39560762fc6a1abbb99d7f40b03367a203e75ce69f8d0d8"}
Aug 04 19:05:24.667654 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:24.667449    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7" event={"ID":"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac","Type":"ContainerStarted","Data":"c3728b1529071f580ac3a387d336fb9990e19d5660b5999e5137f97181460073"}
Aug 04 19:05:24.667654 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:24.667552    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:05:24.668865 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:24.668832    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7" event={"ID":"876f5c85-b5db-4626-a267-5aad04146561","Type":"ContainerStarted","Data":"e7de5e71585b8e37b94bfb98c590fa0caa739c87bc2c97ccb8036ff2d00bba03"}
Aug 04 19:05:24.668996 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:24.668884    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7" event={"ID":"876f5c85-b5db-4626-a267-5aad04146561","Type":"ContainerStarted","Data":"93768b21a36ff2f178a18652cae9646b1cf6ee21b40395ac05c938cbc83ac4af"}
Aug 04 19:05:24.668996 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:24.668922    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:05:24.687835 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:24.687792    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7" podStartSLOduration=1.687778048 podStartE2EDuration="1.687778048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:05:24.686882677 +0000 UTC m=+2266.553797795" watchObservedRunningTime="2026-08-04 19:05:24.687778048 +0000 UTC m=+2266.554693148"
Aug 04 19:05:24.705222 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:24.705180    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7" podStartSLOduration=1.7051675419999999 podStartE2EDuration="1.705167542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:05:24.703962301 +0000 UTC m=+2266.570877402" watchObservedRunningTime="2026-08-04 19:05:24.705167542 +0000 UTC m=+2266.572082642"
Aug 04 19:05:35.675661 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:35.675628    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:05:35.676159 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:35.675704    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:05:36.688864 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:36.688813    2578 patch_prober.go:28] interesting pod/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Aug 04 19:05:36.689363 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:36.688884    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Aug 04 19:05:37.731528 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:37.731493    2578 generic.go:356] "Generic (PLEG): container finished" podID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerID="0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342" exitCode=0
Aug 04 19:05:37.731960 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:37.731551    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" event={"ID":"5f21e5d5-8611-439f-9c86-1f5162f8ab55","Type":"ContainerDied","Data":"0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342"}
Aug 04 19:05:37.732007 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:37.731966    2578 scope.go:117] "RemoveContainer" containerID="0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342"
Aug 04 19:05:37.732963 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:37.732932    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:05:37.733034 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:37.733016    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:05:37.733090 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:37.733076    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:05:38.739788 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:38.739758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" event={"ID":"5f21e5d5-8611-439f-9c86-1f5162f8ab55","Type":"ContainerStarted","Data":"725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994"}
Aug 04 19:05:39.726598 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:39.726545    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:39.726938 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:39.726916    2578 patch_prober.go:28] interesting pod/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Aug 04 19:05:39.727110 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:39.727085    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Aug 04 19:05:40.726513 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:40.726466    2578 patch_prober.go:28] interesting pod/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Aug 04 19:05:40.727030 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:40.726539    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Aug 04 19:05:41.726863 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:41.726824    2578 patch_prober.go:28] interesting pod/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Aug 04 19:05:41.727354 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:41.726897    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Aug 04 19:05:42.726767 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:42.726727    2578 patch_prober.go:28] interesting pod/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Aug 04 19:05:42.727013 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:42.726800    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Aug 04 19:05:43.726033 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:43.725989    2578 patch_prober.go:28] interesting pod/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Aug 04 19:05:43.726238 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:43.726057    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Aug 04 19:05:44.396904 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:44.396864    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:05:44.729844 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:44.729771    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:44.730035 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:44.730014    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:44.730739 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:44.730718    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:05:51.659595 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:05:51.659523    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:06:04.282953 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:04.282914    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:06:41.970905 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:41.970830    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:06:54.571026 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:54.570988    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:06:56.877433 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:56.877397    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:06:56.914036 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:56.914002    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"]
Aug 04 19:06:56.914501 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:56.914468    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7" podUID="876f5c85-b5db-4626-a267-5aad04146561" containerName="payload-processing" containerID="cri-o://e7de5e71585b8e37b94bfb98c590fa0caa739c87bc2c97ccb8036ff2d00bba03" gracePeriod=30
Aug 04 19:06:56.925063 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:56.925031    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"]
Aug 04 19:06:56.925328 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:56.925298    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7" podUID="a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac" containerName="payload-pre-processing" containerID="cri-o://401fcc06fc6f4279b39560762fc6a1abbb99d7f40b03367a203e75ce69f8d0d8" gracePeriod=30
Aug 04 19:06:56.966865 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:56.966833    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:06:57.037593 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.037487    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:06:57.121375 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.121302    2578 generic.go:356] "Generic (PLEG): container finished" podID="a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac" containerID="401fcc06fc6f4279b39560762fc6a1abbb99d7f40b03367a203e75ce69f8d0d8" exitCode=0
Aug 04 19:06:57.121559 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.121541    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7" event={"ID":"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac","Type":"ContainerDied","Data":"401fcc06fc6f4279b39560762fc6a1abbb99d7f40b03367a203e75ce69f8d0d8"}
Aug 04 19:06:57.124136 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.124105    2578 generic.go:356] "Generic (PLEG): container finished" podID="876f5c85-b5db-4626-a267-5aad04146561" containerID="e7de5e71585b8e37b94bfb98c590fa0caa739c87bc2c97ccb8036ff2d00bba03" exitCode=0
Aug 04 19:06:57.124334 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.124175    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7" event={"ID":"876f5c85-b5db-4626-a267-5aad04146561","Type":"ContainerDied","Data":"e7de5e71585b8e37b94bfb98c590fa0caa739c87bc2c97ccb8036ff2d00bba03"}
Aug 04 19:06:57.128263 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.128196    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:06:57.192954 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.192927    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:06:57.226011 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.225985    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:06:57.304711 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.304682    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/876f5c85-b5db-4626-a267-5aad04146561-plugins-config-volume\") pod \"876f5c85-b5db-4626-a267-5aad04146561\" (UID: \"876f5c85-b5db-4626-a267-5aad04146561\") "
Aug 04 19:06:57.304918 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.304726    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-plugins-config-volume\") pod \"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac\" (UID: \"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac\") "
Aug 04 19:06:57.304918 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.304748    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws8bn\" (UniqueName: \"kubernetes.io/projected/876f5c85-b5db-4626-a267-5aad04146561-kube-api-access-ws8bn\") pod \"876f5c85-b5db-4626-a267-5aad04146561\" (UID: \"876f5c85-b5db-4626-a267-5aad04146561\") "
Aug 04 19:06:57.304918 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.304775    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxbf8\" (UniqueName: \"kubernetes.io/projected/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-kube-api-access-gxbf8\") pod \"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac\" (UID: \"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac\") "
Aug 04 19:06:57.305172 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.305146    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/876f5c85-b5db-4626-a267-5aad04146561-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "876f5c85-b5db-4626-a267-5aad04146561" (UID: "876f5c85-b5db-4626-a267-5aad04146561"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:06:57.305235 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.305201    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac" (UID: "a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:06:57.306887 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.306853    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-kube-api-access-gxbf8" (OuterVolumeSpecName: "kube-api-access-gxbf8") pod "a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac" (UID: "a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac"). InnerVolumeSpecName "kube-api-access-gxbf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:06:57.306971 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.306895    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/876f5c85-b5db-4626-a267-5aad04146561-kube-api-access-ws8bn" (OuterVolumeSpecName: "kube-api-access-ws8bn") pod "876f5c85-b5db-4626-a267-5aad04146561" (UID: "876f5c85-b5db-4626-a267-5aad04146561"). InnerVolumeSpecName "kube-api-access-ws8bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:06:57.406440 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.406363    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:06:57.406440 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.406389    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws8bn\" (UniqueName: \"kubernetes.io/projected/876f5c85-b5db-4626-a267-5aad04146561-kube-api-access-ws8bn\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:06:57.406440 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.406399    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxbf8\" (UniqueName: \"kubernetes.io/projected/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac-kube-api-access-gxbf8\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:06:57.406440 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:57.406410    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/876f5c85-b5db-4626-a267-5aad04146561-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:06:58.130907 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.130879    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"
Aug 04 19:06:58.130907 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.130895    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7" event={"ID":"876f5c85-b5db-4626-a267-5aad04146561","Type":"ContainerDied","Data":"93768b21a36ff2f178a18652cae9646b1cf6ee21b40395ac05c938cbc83ac4af"}
Aug 04 19:06:58.131410 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.130953    2578 scope.go:117] "RemoveContainer" containerID="e7de5e71585b8e37b94bfb98c590fa0caa739c87bc2c97ccb8036ff2d00bba03"
Aug 04 19:06:58.132357 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.132337    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7" event={"ID":"a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac","Type":"ContainerDied","Data":"c3728b1529071f580ac3a387d336fb9990e19d5660b5999e5137f97181460073"}
Aug 04 19:06:58.132425 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.132372    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"
Aug 04 19:06:58.139550 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.139531    2578 scope.go:117] "RemoveContainer" containerID="401fcc06fc6f4279b39560762fc6a1abbb99d7f40b03367a203e75ce69f8d0d8"
Aug 04 19:06:58.171807 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.171774    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"]
Aug 04 19:06:58.174317 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.174224    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-59f3c9c8-6786cc67bd-ltzz7"]
Aug 04 19:06:58.198294 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.198221    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"]
Aug 04 19:06:58.205191 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.205167    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-59f3c9c8-77755bc786-s82m7"]
Aug 04 19:06:58.803137 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.803104    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="876f5c85-b5db-4626-a267-5aad04146561" path="/var/lib/kubelet/pods/876f5c85-b5db-4626-a267-5aad04146561/volumes"
Aug 04 19:06:58.803463 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:06:58.803451    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac" path="/var/lib/kubelet/pods/a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac/volumes"
Aug 04 19:07:00.248931 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:00.248890    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:07:05.310022 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:05.309958    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:07:42.718328 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:42.718291    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:07:42.913758 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:42.913719    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:07:49.154340 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:49.154310    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:07:49.242708 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:49.242674    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:07:49.417775 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:49.417702    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"]
Aug 04 19:07:49.417961 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:49.417939    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy" containerID="cri-o://725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994" gracePeriod=30
Aug 04 19:07:49.444118 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:49.444092    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:07:49.668352 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:49.668260    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:07:54.576543 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.576516    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:07:54.706024 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.705954    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-credential-socket\") pod \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") "
Aug 04 19:07:54.706024 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.705985    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-socket\") pod \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") "
Aug 04 19:07:54.706024 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706012    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istiod-ca-cert\") pod \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") "
Aug 04 19:07:54.706301 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706046    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hngxn\" (UniqueName: \"kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-kube-api-access-hngxn\") pod \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") "
Aug 04 19:07:54.706301 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706065    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-podinfo\") pod \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") "
Aug 04 19:07:54.706301 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706091    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-envoy\") pod \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") "
Aug 04 19:07:54.706301 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706125    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-data\") pod \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") "
Aug 04 19:07:54.706301 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706147    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-token\") pod \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") "
Aug 04 19:07:54.706301 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706187    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-certs\") pod \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\" (UID: \"5f21e5d5-8611-439f-9c86-1f5162f8ab55\") "
Aug 04 19:07:54.706301 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706229    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f21e5d5-8611-439f-9c86-1f5162f8ab55" (UID: "5f21e5d5-8611-439f-9c86-1f5162f8ab55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:07:54.706301 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706243    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f21e5d5-8611-439f-9c86-1f5162f8ab55" (UID: "5f21e5d5-8611-439f-9c86-1f5162f8ab55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:07:54.706706 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706383    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f21e5d5-8611-439f-9c86-1f5162f8ab55" (UID: "5f21e5d5-8611-439f-9c86-1f5162f8ab55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:07:54.706706 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706488    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-credential-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:07:54.706706 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706507    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:07:54.706706 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706510    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f21e5d5-8611-439f-9c86-1f5162f8ab55" (UID: "5f21e5d5-8611-439f-9c86-1f5162f8ab55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:07:54.706706 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706522    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-data\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:07:54.706706 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.706554    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f21e5d5-8611-439f-9c86-1f5162f8ab55" (UID: "5f21e5d5-8611-439f-9c86-1f5162f8ab55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:07:54.708236 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.708217    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f21e5d5-8611-439f-9c86-1f5162f8ab55" (UID: "5f21e5d5-8611-439f-9c86-1f5162f8ab55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:07:54.708425 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.708403    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-kube-api-access-hngxn" (OuterVolumeSpecName: "kube-api-access-hngxn") pod "5f21e5d5-8611-439f-9c86-1f5162f8ab55" (UID: "5f21e5d5-8611-439f-9c86-1f5162f8ab55"). InnerVolumeSpecName "kube-api-access-hngxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:07:54.708660 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.708640    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f21e5d5-8611-439f-9c86-1f5162f8ab55" (UID: "5f21e5d5-8611-439f-9c86-1f5162f8ab55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 19:07:54.708729 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.708661    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f21e5d5-8611-439f-9c86-1f5162f8ab55" (UID: "5f21e5d5-8611-439f-9c86-1f5162f8ab55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:07:54.807381 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.807350    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istiod-ca-cert\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:07:54.807381 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.807380    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hngxn\" (UniqueName: \"kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-kube-api-access-hngxn\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:07:54.807552 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.807396    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-podinfo\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:07:54.807552 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.807410    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-envoy\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:07:54.807552 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.807423    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f21e5d5-8611-439f-9c86-1f5162f8ab55-istio-token\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:07:54.807552 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:54.807436    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f21e5d5-8611-439f-9c86-1f5162f8ab55-workload-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:07:55.401714 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.401684    2578 generic.go:356] "Generic (PLEG): container finished" podID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerID="725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994" exitCode=0
Aug 04 19:07:55.401879 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.401745    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"
Aug 04 19:07:55.401879 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.401759    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" event={"ID":"5f21e5d5-8611-439f-9c86-1f5162f8ab55","Type":"ContainerDied","Data":"725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994"}
Aug 04 19:07:55.401879 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.401804    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7" event={"ID":"5f21e5d5-8611-439f-9c86-1f5162f8ab55","Type":"ContainerDied","Data":"081516a1f31310f476d6e9c087e499ffaf7e4c61b7b2febfc34d66a47ba34596"}
Aug 04 19:07:55.401879 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.401825    2578 scope.go:117] "RemoveContainer" containerID="725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994"
Aug 04 19:07:55.410124 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.410107    2578 scope.go:117] "RemoveContainer" containerID="0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342"
Aug 04 19:07:55.418962 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.418945    2578 scope.go:117] "RemoveContainer" containerID="725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994"
Aug 04 19:07:55.419252 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:07:55.419222    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994\": container with ID starting with 725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994 not found: ID does not exist" containerID="725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994"
Aug 04 19:07:55.419312 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.419270    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994"} err="failed to get container status \"725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994\": rpc error: code = NotFound desc = could not find container \"725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994\": container with ID starting with 725377a4ee983aa8e90a3fc49ef84b1b10e62fe6348ac926b5810daf485ab994 not found: ID does not exist"
Aug 04 19:07:55.419312 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.419287    2578 scope.go:117] "RemoveContainer" containerID="0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342"
Aug 04 19:07:55.419529 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:07:55.419514    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342\": container with ID starting with 0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342 not found: ID does not exist" containerID="0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342"
Aug 04 19:07:55.419594 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.419532    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342"} err="failed to get container status \"0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342\": rpc error: code = NotFound desc = could not find container \"0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342\": container with ID starting with 0ab432704512957b8b74988a280e4f71df0d3789a905b7c799587da479d04342 not found: ID does not exist"
Aug 04 19:07:55.422409 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.422388    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"]
Aug 04 19:07:55.426612 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:55.426593    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-59f3c9c8-openshift-default-5c9478fbd4-clhs7"]
Aug 04 19:07:56.802068 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:07:56.802035    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" path="/var/lib/kubelet/pods/5f21e5d5-8611-439f-9c86-1f5162f8ab55/volumes"
Aug 04 19:08:00.960696 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:00.960608    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:05.936360 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:05.936321    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:10.439356 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:10.439315    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:10.538772 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:10.538737    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:11.134358 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:11.134318    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:18.556703 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:18.556663    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:18.661734 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:18.661704    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:18.754995 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:18.754959    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:29.322405 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:29.322371    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:29.519745 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:29.519703    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:50.443484 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:50.443451    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:57.014078 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:57.014037    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:57.115854 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:57.115816    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:57.364988 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:57.364956    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:57.698465 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:57.698381    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:08:57.964673 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:08:57.964563    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:01.563766 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:01.563725    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:01.664096 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:01.664065    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:01.999927 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:01.999842    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:02.127602 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.127552    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:02.241444 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.241414    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:02.285610 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.285529    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"]
Aug 04 19:09:02.285894 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.285881    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy"
Aug 04 19:09:02.285936 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.285895    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy"
Aug 04 19:09:02.285936 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.285905    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="876f5c85-b5db-4626-a267-5aad04146561" containerName="payload-processing"
Aug 04 19:09:02.285936 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.285911    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="876f5c85-b5db-4626-a267-5aad04146561" containerName="payload-processing"
Aug 04 19:09:02.285936 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.285921    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy"
Aug 04 19:09:02.285936 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.285926    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy"
Aug 04 19:09:02.285936 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.285936    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac" containerName="payload-pre-processing"
Aug 04 19:09:02.286118 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.285942    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac" containerName="payload-pre-processing"
Aug 04 19:09:02.286118 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.286011    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="876f5c85-b5db-4626-a267-5aad04146561" containerName="payload-processing"
Aug 04 19:09:02.286118 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.286018    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy"
Aug 04 19:09:02.286118 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.286025    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a00d9ba8-6a03-47ad-9bd8-8b275dcc6cac" containerName="payload-pre-processing"
Aug 04 19:09:02.286118 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.286031    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f21e5d5-8611-439f-9c86-1f5162f8ab55" containerName="istio-proxy"
Aug 04 19:09:02.288932 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.288917    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:09:02.294162 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.294145    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-91f5f5\""
Aug 04 19:09:02.308675 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.308321    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"]
Aug 04 19:09:02.340356 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.340332    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"]
Aug 04 19:09:02.343623 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.343608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:09:02.346671 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.346650    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-91f5f5-dockercfg-n9nr7\""
Aug 04 19:09:02.369539 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.369518    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"]
Aug 04 19:09:02.383187 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.383158    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn5z5\" (UniqueName: \"kubernetes.io/projected/aafca212-f546-4602-a143-ae064f11529e-kube-api-access-dn5z5\") pod \"payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg\" (UID: \"aafca212-f546-4602-a143-ae064f11529e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:09:02.383326 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.383307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aafca212-f546-4602-a143-ae064f11529e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg\" (UID: \"aafca212-f546-4602-a143-ae064f11529e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:09:02.484020 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.483983    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn5z5\" (UniqueName: \"kubernetes.io/projected/aafca212-f546-4602-a143-ae064f11529e-kube-api-access-dn5z5\") pod \"payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg\" (UID: \"aafca212-f546-4602-a143-ae064f11529e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:09:02.484020 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.484024    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4gbc\" (UniqueName: \"kubernetes.io/projected/1c4c6ba3-6215-4279-b043-71e615a2d6c8-kube-api-access-b4gbc\") pod \"payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh\" (UID: \"1c4c6ba3-6215-4279-b043-71e615a2d6c8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:09:02.484346 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.484081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aafca212-f546-4602-a143-ae064f11529e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg\" (UID: \"aafca212-f546-4602-a143-ae064f11529e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:09:02.484346 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.484164    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4c6ba3-6215-4279-b043-71e615a2d6c8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh\" (UID: \"1c4c6ba3-6215-4279-b043-71e615a2d6c8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:09:02.484864 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.484842    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aafca212-f546-4602-a143-ae064f11529e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg\" (UID: \"aafca212-f546-4602-a143-ae064f11529e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:09:02.505862 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.505840    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn5z5\" (UniqueName: \"kubernetes.io/projected/aafca212-f546-4602-a143-ae064f11529e-kube-api-access-dn5z5\") pod \"payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg\" (UID: \"aafca212-f546-4602-a143-ae064f11529e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:09:02.584537 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.584511    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4gbc\" (UniqueName: \"kubernetes.io/projected/1c4c6ba3-6215-4279-b043-71e615a2d6c8-kube-api-access-b4gbc\") pod \"payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh\" (UID: \"1c4c6ba3-6215-4279-b043-71e615a2d6c8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:09:02.584878 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.584614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4c6ba3-6215-4279-b043-71e615a2d6c8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh\" (UID: \"1c4c6ba3-6215-4279-b043-71e615a2d6c8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:09:02.585220 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.585202    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4c6ba3-6215-4279-b043-71e615a2d6c8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh\" (UID: \"1c4c6ba3-6215-4279-b043-71e615a2d6c8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:09:02.592970 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.592946    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4gbc\" (UniqueName: \"kubernetes.io/projected/1c4c6ba3-6215-4279-b043-71e615a2d6c8-kube-api-access-b4gbc\") pod \"payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh\" (UID: \"1c4c6ba3-6215-4279-b043-71e615a2d6c8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:09:02.598744 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.598728    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:09:02.653166 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.653134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:09:02.750530 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.750505    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"]
Aug 04 19:09:02.751952 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:09:02.751925    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c4c6ba3_6215_4279_b043_71e615a2d6c8.slice/crio-46ade31b7a421fbec415a89137641cbb832cae5ef09bc167a795e38e437e1fdb WatchSource:0}: Error finding container 46ade31b7a421fbec415a89137641cbb832cae5ef09bc167a795e38e437e1fdb: Status 404 returned error can't find the container with id 46ade31b7a421fbec415a89137641cbb832cae5ef09bc167a795e38e437e1fdb
Aug 04 19:09:02.808114 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:02.808054    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"]
Aug 04 19:09:02.809967 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:09:02.809942    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaafca212_f546_4602_a143_ae064f11529e.slice/crio-9d3f027988fe0c32e84692e681f28f3191f6dc5531a2bbe36bcecf5d877cf353 WatchSource:0}: Error finding container 9d3f027988fe0c32e84692e681f28f3191f6dc5531a2bbe36bcecf5d877cf353: Status 404 returned error can't find the container with id 9d3f027988fe0c32e84692e681f28f3191f6dc5531a2bbe36bcecf5d877cf353
Aug 04 19:09:03.732276 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:03.732243    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh" event={"ID":"1c4c6ba3-6215-4279-b043-71e615a2d6c8","Type":"ContainerStarted","Data":"0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565"}
Aug 04 19:09:03.732743 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:03.732302    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh" event={"ID":"1c4c6ba3-6215-4279-b043-71e615a2d6c8","Type":"ContainerStarted","Data":"46ade31b7a421fbec415a89137641cbb832cae5ef09bc167a795e38e437e1fdb"}
Aug 04 19:09:03.732743 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:03.732332    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:09:03.733819 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:03.733794    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg" event={"ID":"aafca212-f546-4602-a143-ae064f11529e","Type":"ContainerStarted","Data":"4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f"}
Aug 04 19:09:03.733927 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:03.733835    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg" event={"ID":"aafca212-f546-4602-a143-ae064f11529e","Type":"ContainerStarted","Data":"9d3f027988fe0c32e84692e681f28f3191f6dc5531a2bbe36bcecf5d877cf353"}
Aug 04 19:09:03.733927 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:03.733866    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:09:03.752141 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:03.752101    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh" podStartSLOduration=1.7520875729999998 podStartE2EDuration="1.752087573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:09:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:09:03.751298645 +0000 UTC m=+2485.618213758" watchObservedRunningTime="2026-08-04 19:09:03.752087573 +0000 UTC m=+2485.619002673"
Aug 04 19:09:03.770534 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:03.770490    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg" podStartSLOduration=1.770476669 podStartE2EDuration="1.770476669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:09:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:09:03.769004304 +0000 UTC m=+2485.635919404" watchObservedRunningTime="2026-08-04 19:09:03.770476669 +0000 UTC m=+2485.637391769"
Aug 04 19:09:14.740798 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:14.740760    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:09:14.741170 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:14.740966    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:09:25.311647 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.311607    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:25.616194 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.616083    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:25.915950 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.915855    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"]
Aug 04 19:09:25.919421 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.919398    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:25.922567 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.922549    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-77907c-openshift-default-dockercfg-dfpj4\""
Aug 04 19:09:25.937640 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.937618    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"]
Aug 04 19:09:25.947911 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.947885    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:25.971831 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.971808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b214aab8-ad7e-4482-978d-1efc7e68294b-istiod-ca-cert\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:25.971925 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.971838    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-socket\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:25.971925 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.971858    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-token\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:25.972000 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.971937    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-credential-socket\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:25.972034 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.971996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7h5p\" (UniqueName: \"kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-kube-api-access-d7h5p\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:25.972068 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.972045    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-envoy\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:25.972101 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.972070    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-data\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:25.972134 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.972095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-certs\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:25.972168 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:25.972135    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-podinfo\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.072535 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.072501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-certs\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.072714 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.072541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-podinfo\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.072714 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.072565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b214aab8-ad7e-4482-978d-1efc7e68294b-istiod-ca-cert\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.072714 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.072610    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-socket\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.072714 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.072630    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-token\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.072904 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.072707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-credential-socket\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.072904 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.072751    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7h5p\" (UniqueName: \"kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-kube-api-access-d7h5p\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.072904 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.072828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-envoy\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.073129 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.073111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-data\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.073201 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.072923    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-certs\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.073201 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.072974    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-socket\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.073282 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.073197    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-credential-socket\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.073378 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.073352    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b214aab8-ad7e-4482-978d-1efc7e68294b-istiod-ca-cert\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.073487 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.073417    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-data\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.074920 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.074897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-envoy\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.075120 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.075106    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-podinfo\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.083739 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.083718    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7h5p\" (UniqueName: \"kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-kube-api-access-d7h5p\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.084193 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.084161    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-token\") pod \"e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") " pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.230391 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.230321    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:26.372543 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.372509    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:26.384599 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.384560    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"]
Aug 04 19:09:26.387508 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:09:26.387466    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb214aab8_ad7e_4482_978d_1efc7e68294b.slice/crio-cc3d6627821064e49e2f98f63420bd77f70e8d6ce39ca226b1e34c41ee6e50b3 WatchSource:0}: Error finding container cc3d6627821064e49e2f98f63420bd77f70e8d6ce39ca226b1e34c41ee6e50b3: Status 404 returned error can't find the container with id cc3d6627821064e49e2f98f63420bd77f70e8d6ce39ca226b1e34c41ee6e50b3
Aug 04 19:09:26.390538 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.390499    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:09:26.390646 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.390616    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:09:26.390691 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.390662    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:09:26.838519 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.838484    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" event={"ID":"b214aab8-ad7e-4482-978d-1efc7e68294b","Type":"ContainerStarted","Data":"2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb"}
Aug 04 19:09:26.838708 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.838538    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" event={"ID":"b214aab8-ad7e-4482-978d-1efc7e68294b","Type":"ContainerStarted","Data":"cc3d6627821064e49e2f98f63420bd77f70e8d6ce39ca226b1e34c41ee6e50b3"}
Aug 04 19:09:26.873753 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:26.873704    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" podStartSLOduration=1.873688815 podStartE2EDuration="1.873688815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:09:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:09:26.872464007 +0000 UTC m=+2508.739379107" watchObservedRunningTime="2026-08-04 19:09:26.873688815 +0000 UTC m=+2508.740603914"
Aug 04 19:09:27.231174 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:27.231093    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:27.236029 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:27.236008    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:27.843571 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:27.843539    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:27.844459 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:27.844441    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:29.611473 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:29.611437    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:29.730393 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:29.730364    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:30.002568 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:30.002475    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:30.200889 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:30.200852    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:30.775267 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:30.775178    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:30.881715 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:30.881678    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:43.177159 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:43.177125    2578 patch_prober.go:28] interesting pod/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": read tcp 10.132.0.2:49464->10.132.0.50:15021: read: connection reset by peer" start-of-body=
Aug 04 19:09:43.177525 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:43.177189    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": read tcp 10.132.0.2:49464->10.132.0.50:15021: read: connection reset by peer"
Aug 04 19:09:43.916917 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:43.916888    2578 generic.go:356] "Generic (PLEG): container finished" podID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerID="2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb" exitCode=0
Aug 04 19:09:43.917084 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:43.916939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" event={"ID":"b214aab8-ad7e-4482-978d-1efc7e68294b","Type":"ContainerDied","Data":"2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb"}
Aug 04 19:09:43.917261 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:43.917249    2578 scope.go:117] "RemoveContainer" containerID="2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb"
Aug 04 19:09:43.918187 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:43.918154    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:09:43.918287 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:43.918237    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:09:43.918352 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:43.918286    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:09:44.928053 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:44.928007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" event={"ID":"b214aab8-ad7e-4482-978d-1efc7e68294b","Type":"ContainerStarted","Data":"5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea"}
Aug 04 19:09:45.231450 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:45.231350    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:45.231648 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:45.231463    2578 patch_prober.go:28] interesting pod/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Aug 04 19:09:45.231648 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:45.231520    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Aug 04 19:09:46.232219 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:46.231757    2578 patch_prober.go:28] interesting pod/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Aug 04 19:09:46.232219 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:46.231833    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Aug 04 19:09:47.231302 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:47.231255    2578 patch_prober.go:28] interesting pod/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Aug 04 19:09:47.231507 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:47.231323    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Aug 04 19:09:48.231213 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:48.231170    2578 patch_prober.go:28] interesting pod/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Aug 04 19:09:48.231704 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:48.231244    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Aug 04 19:09:49.230924 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:49.230882    2578 patch_prober.go:28] interesting pod/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Aug 04 19:09:49.231128 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:49.230957    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Aug 04 19:09:50.233907 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:50.233878    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:50.234292 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:50.234243    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:50.234977 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:50.234957    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:09:53.826597 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:53.826542    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:57.737271 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.737240    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"]
Aug 04 19:09:57.740998 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.740979    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.749141 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.749122    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-91f5f5\"/\"test-model-91f5f5-kserve-self-signed-certs\""
Aug 04 19:09:57.750223 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.750204    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-91f5f5\"/\"kube-root-ca.crt\""
Aug 04 19:09:57.750223 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.750216    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-91f5f5\"/\"openshift-service-ca.crt\""
Aug 04 19:09:57.750749 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.750732    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-91f5f5\"/\"default-dockercfg-pjdww\""
Aug 04 19:09:57.768312 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.768284    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:09:57.770513 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.770493    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"]
Aug 04 19:09:57.856704 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.856678    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2edcadee-d94f-4fd7-92cd-781711d5523f-tls-certs\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.856859 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.856708    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-model-cache\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.856859 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.856750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-dshm\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.856859 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.856770    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-tmp-dir\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.856859 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.856799    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-home\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.856992 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.856896    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdf7r\" (UniqueName: \"kubernetes.io/projected/2edcadee-d94f-4fd7-92cd-781711d5523f-kube-api-access-fdf7r\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.958057 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.958029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdf7r\" (UniqueName: \"kubernetes.io/projected/2edcadee-d94f-4fd7-92cd-781711d5523f-kube-api-access-fdf7r\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.958256 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.958073    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2edcadee-d94f-4fd7-92cd-781711d5523f-tls-certs\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.958256 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.958101    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-model-cache\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.958256 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.958153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-dshm\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.958256 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.958183    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-tmp-dir\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.958256 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.958244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-home\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.958698 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.958675    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-home\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.958791 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.958673    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-model-cache\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.958845 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.958826    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-tmp-dir\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.960500 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.960476    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2edcadee-d94f-4fd7-92cd-781711d5523f-tls-certs\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.960500 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.960478    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-dshm\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:57.968996 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:57.968970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdf7r\" (UniqueName: \"kubernetes.io/projected/2edcadee-d94f-4fd7-92cd-781711d5523f-kube-api-access-fdf7r\") pod \"test-model-91f5f5-kserve-6d867dcf65-6r9vs\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") " pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:58.050543 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:58.050519    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:09:58.191547 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:58.191526    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"]
Aug 04 19:09:58.193956 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:09:58.193930    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2edcadee_d94f_4fd7_92cd_781711d5523f.slice/crio-cc787015e188aacff5bcb9ceb9d7e304bcb0ae959addfa6f889b68f7dfb89b3c WatchSource:0}: Error finding container cc787015e188aacff5bcb9ceb9d7e304bcb0ae959addfa6f889b68f7dfb89b3c: Status 404 returned error can't find the container with id cc787015e188aacff5bcb9ceb9d7e304bcb0ae959addfa6f889b68f7dfb89b3c
Aug 04 19:09:58.997347 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:58.997318    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs" event={"ID":"2edcadee-d94f-4fd7-92cd-781711d5523f","Type":"ContainerStarted","Data":"7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3"}
Aug 04 19:09:58.997702 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:58.997357    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs" event={"ID":"2edcadee-d94f-4fd7-92cd-781711d5523f","Type":"ContainerStarted","Data":"cc787015e188aacff5bcb9ceb9d7e304bcb0ae959addfa6f889b68f7dfb89b3c"}
Aug 04 19:09:59.032014 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:09:59.031966    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs" podStartSLOduration=2.031950609 podStartE2EDuration="2.031950609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:09:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:09:59.031331375 +0000 UTC m=+2540.898246479" watchObservedRunningTime="2026-08-04 19:09:59.031950609 +0000 UTC m=+2540.898865708"
Aug 04 19:10:08.051654 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:10:08.051620    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:10:08.051994 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:10:08.051670    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:10:08.063723 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:10:08.063703    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:10:09.054894 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:10:09.054861    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:10:32.928306 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:10:32.928272    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:13.749087 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:13.749012    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"]
Aug 04 19:11:13.771761 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:13.771735    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:14.713895 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:14.713851    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:25.338297 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.338263    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:25.364209 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.364181    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"]
Aug 04 19:11:25.364473 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.364401    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh" podUID="1c4c6ba3-6215-4279-b043-71e615a2d6c8" containerName="payload-processing" containerID="cri-o://0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565" gracePeriod=30
Aug 04 19:11:25.376826 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.376800    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"]
Aug 04 19:11:25.377044 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.377021    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg" podUID="aafca212-f546-4602-a143-ae064f11529e" containerName="payload-pre-processing" containerID="cri-o://4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f" gracePeriod=30
Aug 04 19:11:25.439926 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.439897    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:25.542870 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.540545    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:25.658723 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.658691    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:25.663502 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.663475    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:11:25.666388 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.666373    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:11:25.804827 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.804798    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4gbc\" (UniqueName: \"kubernetes.io/projected/1c4c6ba3-6215-4279-b043-71e615a2d6c8-kube-api-access-b4gbc\") pod \"1c4c6ba3-6215-4279-b043-71e615a2d6c8\" (UID: \"1c4c6ba3-6215-4279-b043-71e615a2d6c8\") "
Aug 04 19:11:25.804827 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.804830    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aafca212-f546-4602-a143-ae064f11529e-plugins-config-volume\") pod \"aafca212-f546-4602-a143-ae064f11529e\" (UID: \"aafca212-f546-4602-a143-ae064f11529e\") "
Aug 04 19:11:25.805070 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.804864    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4c6ba3-6215-4279-b043-71e615a2d6c8-plugins-config-volume\") pod \"1c4c6ba3-6215-4279-b043-71e615a2d6c8\" (UID: \"1c4c6ba3-6215-4279-b043-71e615a2d6c8\") "
Aug 04 19:11:25.805070 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.804910    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn5z5\" (UniqueName: \"kubernetes.io/projected/aafca212-f546-4602-a143-ae064f11529e-kube-api-access-dn5z5\") pod \"aafca212-f546-4602-a143-ae064f11529e\" (UID: \"aafca212-f546-4602-a143-ae064f11529e\") "
Aug 04 19:11:25.805228 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.805202    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c4c6ba3-6215-4279-b043-71e615a2d6c8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1c4c6ba3-6215-4279-b043-71e615a2d6c8" (UID: "1c4c6ba3-6215-4279-b043-71e615a2d6c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:11:25.805280 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.805218    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aafca212-f546-4602-a143-ae064f11529e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aafca212-f546-4602-a143-ae064f11529e" (UID: "aafca212-f546-4602-a143-ae064f11529e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:11:25.806917 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.806899    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c4c6ba3-6215-4279-b043-71e615a2d6c8-kube-api-access-b4gbc" (OuterVolumeSpecName: "kube-api-access-b4gbc") pod "1c4c6ba3-6215-4279-b043-71e615a2d6c8" (UID: "1c4c6ba3-6215-4279-b043-71e615a2d6c8"). InnerVolumeSpecName "kube-api-access-b4gbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:11:25.807062 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.807043    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aafca212-f546-4602-a143-ae064f11529e-kube-api-access-dn5z5" (OuterVolumeSpecName: "kube-api-access-dn5z5") pod "aafca212-f546-4602-a143-ae064f11529e" (UID: "aafca212-f546-4602-a143-ae064f11529e"). InnerVolumeSpecName "kube-api-access-dn5z5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:11:25.906407 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.906323    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4gbc\" (UniqueName: \"kubernetes.io/projected/1c4c6ba3-6215-4279-b043-71e615a2d6c8-kube-api-access-b4gbc\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:25.906407 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.906352    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aafca212-f546-4602-a143-ae064f11529e-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:25.906407 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.906364    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4c6ba3-6215-4279-b043-71e615a2d6c8-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:25.906407 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.906377    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn5z5\" (UniqueName: \"kubernetes.io/projected/aafca212-f546-4602-a143-ae064f11529e-kube-api-access-dn5z5\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:25.927620 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:25.927560    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:26.409059 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.409024    2578 generic.go:356] "Generic (PLEG): container finished" podID="1c4c6ba3-6215-4279-b043-71e615a2d6c8" containerID="0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565" exitCode=0
Aug 04 19:11:26.409526 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.409075    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"
Aug 04 19:11:26.409526 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.409111    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh" event={"ID":"1c4c6ba3-6215-4279-b043-71e615a2d6c8","Type":"ContainerDied","Data":"0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565"}
Aug 04 19:11:26.409526 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.409158    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh" event={"ID":"1c4c6ba3-6215-4279-b043-71e615a2d6c8","Type":"ContainerDied","Data":"46ade31b7a421fbec415a89137641cbb832cae5ef09bc167a795e38e437e1fdb"}
Aug 04 19:11:26.409526 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.409186    2578 scope.go:117] "RemoveContainer" containerID="0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565"
Aug 04 19:11:26.410503 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.410429    2578 generic.go:356] "Generic (PLEG): container finished" podID="aafca212-f546-4602-a143-ae064f11529e" containerID="4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f" exitCode=0
Aug 04 19:11:26.410503 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.410470    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"
Aug 04 19:11:26.410656 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.410513    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg" event={"ID":"aafca212-f546-4602-a143-ae064f11529e","Type":"ContainerDied","Data":"4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f"}
Aug 04 19:11:26.410656 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.410554    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg" event={"ID":"aafca212-f546-4602-a143-ae064f11529e","Type":"ContainerDied","Data":"9d3f027988fe0c32e84692e681f28f3191f6dc5531a2bbe36bcecf5d877cf353"}
Aug 04 19:11:26.418269 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.418248    2578 scope.go:117] "RemoveContainer" containerID="0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565"
Aug 04 19:11:26.418915 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:11:26.418503    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565\": container with ID starting with 0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565 not found: ID does not exist" containerID="0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565"
Aug 04 19:11:26.418915 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.418525    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565"} err="failed to get container status \"0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565\": rpc error: code = NotFound desc = could not find container \"0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565\": container with ID starting with 0f98be7b2c8a6330b9d03076854cc2ae8e2729b09527f72aaf6fe9c24ace5565 not found: ID does not exist"
Aug 04 19:11:26.418915 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.418540    2578 scope.go:117] "RemoveContainer" containerID="4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f"
Aug 04 19:11:26.426643 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.426626    2578 scope.go:117] "RemoveContainer" containerID="4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f"
Aug 04 19:11:26.426885 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:11:26.426868    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f\": container with ID starting with 4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f not found: ID does not exist" containerID="4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f"
Aug 04 19:11:26.426931 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.426892    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f"} err="failed to get container status \"4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f\": rpc error: code = NotFound desc = could not find container \"4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f\": container with ID starting with 4f29d16dcdf0d59fd7b28d478c973213f704d43ecce5a704eb565723cfd4875f not found: ID does not exist"
Aug 04 19:11:26.440525 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.440501    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"]
Aug 04 19:11:26.444703 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.444679    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-91f5f5-9df4555b4-x7zbg"]
Aug 04 19:11:26.456062 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.456033    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"]
Aug 04 19:11:26.463427 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.463405    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-91f5f5-6f65678bbb-fcdnh"]
Aug 04 19:11:26.802190 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.802159    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c4c6ba3-6215-4279-b043-71e615a2d6c8" path="/var/lib/kubelet/pods/1c4c6ba3-6215-4279-b043-71e615a2d6c8/volumes"
Aug 04 19:11:26.802520 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:26.802507    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aafca212-f546-4602-a143-ae064f11529e" path="/var/lib/kubelet/pods/aafca212-f546-4602-a143-ae064f11529e/volumes"
Aug 04 19:11:28.767385 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:28.767341    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs" podUID="2edcadee-d94f-4fd7-92cd-781711d5523f" containerName="main" containerID="cri-o://7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3" gracePeriod=45
Aug 04 19:11:29.018950 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.018893    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:11:29.133913 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.133887    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-model-cache\") pod \"2edcadee-d94f-4fd7-92cd-781711d5523f\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") "
Aug 04 19:11:29.134107 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.133922    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdf7r\" (UniqueName: \"kubernetes.io/projected/2edcadee-d94f-4fd7-92cd-781711d5523f-kube-api-access-fdf7r\") pod \"2edcadee-d94f-4fd7-92cd-781711d5523f\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") "
Aug 04 19:11:29.134107 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.133943    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2edcadee-d94f-4fd7-92cd-781711d5523f-tls-certs\") pod \"2edcadee-d94f-4fd7-92cd-781711d5523f\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") "
Aug 04 19:11:29.134107 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.133982    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-tmp-dir\") pod \"2edcadee-d94f-4fd7-92cd-781711d5523f\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") "
Aug 04 19:11:29.134107 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.134089    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-home\") pod \"2edcadee-d94f-4fd7-92cd-781711d5523f\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") "
Aug 04 19:11:29.134330 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.134141    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-dshm\") pod \"2edcadee-d94f-4fd7-92cd-781711d5523f\" (UID: \"2edcadee-d94f-4fd7-92cd-781711d5523f\") "
Aug 04 19:11:29.134330 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.134212    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2edcadee-d94f-4fd7-92cd-781711d5523f" (UID: "2edcadee-d94f-4fd7-92cd-781711d5523f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:11:29.134330 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.134234    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-model-cache" (OuterVolumeSpecName: "model-cache") pod "2edcadee-d94f-4fd7-92cd-781711d5523f" (UID: "2edcadee-d94f-4fd7-92cd-781711d5523f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:11:29.134330 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.134307    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-home" (OuterVolumeSpecName: "home") pod "2edcadee-d94f-4fd7-92cd-781711d5523f" (UID: "2edcadee-d94f-4fd7-92cd-781711d5523f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:11:29.134479 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.134456    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-model-cache\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:29.134558 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.134482    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-tmp-dir\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:29.134558 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.134495    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-home\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:29.136167 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.136136    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-dshm" (OuterVolumeSpecName: "dshm") pod "2edcadee-d94f-4fd7-92cd-781711d5523f" (UID: "2edcadee-d94f-4fd7-92cd-781711d5523f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:11:29.136284 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.136176    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2edcadee-d94f-4fd7-92cd-781711d5523f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2edcadee-d94f-4fd7-92cd-781711d5523f" (UID: "2edcadee-d94f-4fd7-92cd-781711d5523f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 19:11:29.136284 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.136212    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2edcadee-d94f-4fd7-92cd-781711d5523f-kube-api-access-fdf7r" (OuterVolumeSpecName: "kube-api-access-fdf7r") pod "2edcadee-d94f-4fd7-92cd-781711d5523f" (UID: "2edcadee-d94f-4fd7-92cd-781711d5523f"). InnerVolumeSpecName "kube-api-access-fdf7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:11:29.235086 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.235057    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdf7r\" (UniqueName: \"kubernetes.io/projected/2edcadee-d94f-4fd7-92cd-781711d5523f-kube-api-access-fdf7r\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:29.235086 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.235084    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2edcadee-d94f-4fd7-92cd-781711d5523f-tls-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:29.235086 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.235095    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2edcadee-d94f-4fd7-92cd-781711d5523f-dshm\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:29.429532 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.429493    2578 generic.go:356] "Generic (PLEG): container finished" podID="2edcadee-d94f-4fd7-92cd-781711d5523f" containerID="7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3" exitCode=0
Aug 04 19:11:29.429756 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.429540    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"
Aug 04 19:11:29.429756 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.429591    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs" event={"ID":"2edcadee-d94f-4fd7-92cd-781711d5523f","Type":"ContainerDied","Data":"7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3"}
Aug 04 19:11:29.429756 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.429638    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs" event={"ID":"2edcadee-d94f-4fd7-92cd-781711d5523f","Type":"ContainerDied","Data":"cc787015e188aacff5bcb9ceb9d7e304bcb0ae959addfa6f889b68f7dfb89b3c"}
Aug 04 19:11:29.429756 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.429661    2578 scope.go:117] "RemoveContainer" containerID="7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3"
Aug 04 19:11:29.438683 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.438658    2578 scope.go:117] "RemoveContainer" containerID="7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3"
Aug 04 19:11:29.438979 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:11:29.438963    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3\": container with ID starting with 7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3 not found: ID does not exist" containerID="7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3"
Aug 04 19:11:29.439071 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.438985    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3"} err="failed to get container status \"7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3\": rpc error: code = NotFound desc = could not find container \"7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3\": container with ID starting with 7aad9272d51b128fd546bc9ed34a90abd8e5b710b53fdfaebd32fa69e0c468e3 not found: ID does not exist"
Aug 04 19:11:29.450766 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.450740    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"]
Aug 04 19:11:29.459042 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:29.459018    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-91f5f5/test-model-91f5f5-kserve-6d867dcf65-6r9vs"]
Aug 04 19:11:30.802089 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:30.802054    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2edcadee-d94f-4fd7-92cd-781711d5523f" path="/var/lib/kubelet/pods/2edcadee-d94f-4fd7-92cd-781711d5523f/volumes"
Aug 04 19:11:31.525872 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:31.525836    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:32.188803 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:32.188763    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:32.290062 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:32.290024    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:32.450791 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:32.450698    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:32.724266 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:32.724179    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:37.722504 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:37.722476    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:37.815053 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:37.815023    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:37.898281 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:37.898239    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:37.978826 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:37.978734    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:38.379989 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:38.379941    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:44.958191 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:44.958158    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:45.041205 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.041176    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"]
Aug 04 19:11:45.041377 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.041358    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy" containerID="cri-o://5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea" gracePeriod=30
Aug 04 19:11:45.066253 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.066224    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:45.334819 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.334787    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:45.405901 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.405866    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:45.959775 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.959738    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"]
Aug 04 19:11:45.960283 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.960263    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c4c6ba3-6215-4279-b043-71e615a2d6c8" containerName="payload-processing"
Aug 04 19:11:45.960367 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.960286    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c4c6ba3-6215-4279-b043-71e615a2d6c8" containerName="payload-processing"
Aug 04 19:11:45.960367 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.960300    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aafca212-f546-4602-a143-ae064f11529e" containerName="payload-pre-processing"
Aug 04 19:11:45.960367 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.960309    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aafca212-f546-4602-a143-ae064f11529e" containerName="payload-pre-processing"
Aug 04 19:11:45.960367 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.960325    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2edcadee-d94f-4fd7-92cd-781711d5523f" containerName="main"
Aug 04 19:11:45.960367 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.960333    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2edcadee-d94f-4fd7-92cd-781711d5523f" containerName="main"
Aug 04 19:11:45.960701 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.960447    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c4c6ba3-6215-4279-b043-71e615a2d6c8" containerName="payload-processing"
Aug 04 19:11:45.960701 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.960460    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2edcadee-d94f-4fd7-92cd-781711d5523f" containerName="main"
Aug 04 19:11:45.960701 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.960469    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="aafca212-f546-4602-a143-ae064f11529e" containerName="payload-pre-processing"
Aug 04 19:11:45.964696 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.964670    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:45.967923 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.967897    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c028db-openshift-default-dockercfg-6r85x\""
Aug 04 19:11:45.997690 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:45.997652    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"]
Aug 04 19:11:46.080955 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.080920    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:46.082897 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.082876    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-istio-token\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.082988 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.082906    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-credential-socket\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.082988 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.082926    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-data\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.082988 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.082950    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06185225-8261-47ee-9caf-0064672bc414-istiod-ca-cert\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.083086 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.083012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-envoy\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.083086 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.083064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-certs\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.083150 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.083086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06185225-8261-47ee-9caf-0064672bc414-istio-podinfo\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.083150 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.083124    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxnv4\" (UniqueName: \"kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-kube-api-access-fxnv4\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.083150 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.083146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-socket\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.184234 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06185225-8261-47ee-9caf-0064672bc414-istiod-ca-cert\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.184234 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184236    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-envoy\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.184475 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-certs\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.184475 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06185225-8261-47ee-9caf-0064672bc414-istio-podinfo\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.184475 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184316    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxnv4\" (UniqueName: \"kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-kube-api-access-fxnv4\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.184475 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184331    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-socket\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.184475 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184387    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-istio-token\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.184475 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-credential-socket\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.184475 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-data\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.184892 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184864    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-data\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.185070 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.185043    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06185225-8261-47ee-9caf-0064672bc414-istiod-ca-cert\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.185158 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184967    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-certs\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.185158 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184997    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-credential-socket\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.185158 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.184897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-socket\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.186736 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.186705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-envoy\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.186914 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.186897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06185225-8261-47ee-9caf-0064672bc414-istio-podinfo\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.195770 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.195744    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-istio-token\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.197807 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.197788    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxnv4\" (UniqueName: \"kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-kube-api-access-fxnv4\") pod \"e2e-api-a-c028db-openshift-default-5479dcb646-t2j95\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") " pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.200676 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.200645    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:46.274711 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.274628    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:46.307929 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.307898    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:46.423553 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.423529    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"]
Aug 04 19:11:46.425925 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:11:46.425900    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06185225_8261_47ee_9caf_0064672bc414.slice/crio-c284b404f6d2a29d5d79b4e9fffea6e03f6913a3cf392c93443353950d48895a WatchSource:0}: Error finding container c284b404f6d2a29d5d79b4e9fffea6e03f6913a3cf392c93443353950d48895a: Status 404 returned error can't find the container with id c284b404f6d2a29d5d79b4e9fffea6e03f6913a3cf392c93443353950d48895a
Aug 04 19:11:46.427832 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.427797    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:11:46.427894 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.427863    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:11:46.427935 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.427894    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:11:46.510447 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.510417    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" event={"ID":"06185225-8261-47ee-9caf-0064672bc414","Type":"ContainerStarted","Data":"b7a8228b20780d279ac1456e0629e904a65cae209a5212ce55a29ab26a8b501f"}
Aug 04 19:11:46.510626 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.510464    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" event={"ID":"06185225-8261-47ee-9caf-0064672bc414","Type":"ContainerStarted","Data":"c284b404f6d2a29d5d79b4e9fffea6e03f6913a3cf392c93443353950d48895a"}
Aug 04 19:11:46.538705 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:46.538657    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" podStartSLOduration=1.538640997 podStartE2EDuration="1.538640997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:11:46.535602504 +0000 UTC m=+2648.402517605" watchObservedRunningTime="2026-08-04 19:11:46.538640997 +0000 UTC m=+2648.405556097"
Aug 04 19:11:47.275120 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:47.275088    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:47.279999 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:47.279976    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:47.515932 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:47.515904    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:47.516902 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:47.516880    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:11:48.753348 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:48.753311    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:48.902606 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:48.902560    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:49.972279 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:49.972245    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:50.298031 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.298009    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:11:50.318467 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318446    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-credential-socket\") pod \"b214aab8-ad7e-4482-978d-1efc7e68294b\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") "
Aug 04 19:11:50.318545 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318476    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-certs\") pod \"b214aab8-ad7e-4482-978d-1efc7e68294b\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") "
Aug 04 19:11:50.318545 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318497    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-podinfo\") pod \"b214aab8-ad7e-4482-978d-1efc7e68294b\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") "
Aug 04 19:11:50.318655 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318563    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-envoy\") pod \"b214aab8-ad7e-4482-978d-1efc7e68294b\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") "
Aug 04 19:11:50.318655 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318623    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-data\") pod \"b214aab8-ad7e-4482-978d-1efc7e68294b\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") "
Aug 04 19:11:50.318742 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318654    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-socket\") pod \"b214aab8-ad7e-4482-978d-1efc7e68294b\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") "
Aug 04 19:11:50.318742 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318682    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7h5p\" (UniqueName: \"kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-kube-api-access-d7h5p\") pod \"b214aab8-ad7e-4482-978d-1efc7e68294b\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") "
Aug 04 19:11:50.318742 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318708    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b214aab8-ad7e-4482-978d-1efc7e68294b-istiod-ca-cert\") pod \"b214aab8-ad7e-4482-978d-1efc7e68294b\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") "
Aug 04 19:11:50.318898 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318743    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-token\") pod \"b214aab8-ad7e-4482-978d-1efc7e68294b\" (UID: \"b214aab8-ad7e-4482-978d-1efc7e68294b\") "
Aug 04 19:11:50.318898 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318746    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b214aab8-ad7e-4482-978d-1efc7e68294b" (UID: "b214aab8-ad7e-4482-978d-1efc7e68294b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:11:50.318898 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318759    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b214aab8-ad7e-4482-978d-1efc7e68294b" (UID: "b214aab8-ad7e-4482-978d-1efc7e68294b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:11:50.319068 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318937    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-data" (OuterVolumeSpecName: "istio-data") pod "b214aab8-ad7e-4482-978d-1efc7e68294b" (UID: "b214aab8-ad7e-4482-978d-1efc7e68294b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:11:50.319068 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.318960    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b214aab8-ad7e-4482-978d-1efc7e68294b" (UID: "b214aab8-ad7e-4482-978d-1efc7e68294b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:11:50.319068 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.319060    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-data\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:50.319068 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.319063    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b214aab8-ad7e-4482-978d-1efc7e68294b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b214aab8-ad7e-4482-978d-1efc7e68294b" (UID: "b214aab8-ad7e-4482-978d-1efc7e68294b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:11:50.319240 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.319077    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:50.319240 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.319092    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-credential-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:50.319240 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.319106    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-workload-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:50.320604 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.320556    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b214aab8-ad7e-4482-978d-1efc7e68294b" (UID: "b214aab8-ad7e-4482-978d-1efc7e68294b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 19:11:50.320692 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.320670    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-kube-api-access-d7h5p" (OuterVolumeSpecName: "kube-api-access-d7h5p") pod "b214aab8-ad7e-4482-978d-1efc7e68294b" (UID: "b214aab8-ad7e-4482-978d-1efc7e68294b"). InnerVolumeSpecName "kube-api-access-d7h5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:11:50.320958 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.320941    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b214aab8-ad7e-4482-978d-1efc7e68294b" (UID: "b214aab8-ad7e-4482-978d-1efc7e68294b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:11:50.321024 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.321006    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-token" (OuterVolumeSpecName: "istio-token") pod "b214aab8-ad7e-4482-978d-1efc7e68294b" (UID: "b214aab8-ad7e-4482-978d-1efc7e68294b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:11:50.420468 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.420431    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-token\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:50.420468 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.420475    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-podinfo\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:50.420694 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.420490    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b214aab8-ad7e-4482-978d-1efc7e68294b-istio-envoy\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:50.420694 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.420504    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7h5p\" (UniqueName: \"kubernetes.io/projected/b214aab8-ad7e-4482-978d-1efc7e68294b-kube-api-access-d7h5p\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:50.420694 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.420518    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b214aab8-ad7e-4482-978d-1efc7e68294b-istiod-ca-cert\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:11:50.532140 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.532040    2578 generic.go:356] "Generic (PLEG): container finished" podID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerID="5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea" exitCode=0
Aug 04 19:11:50.532140 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.532086    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" event={"ID":"b214aab8-ad7e-4482-978d-1efc7e68294b","Type":"ContainerDied","Data":"5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea"}
Aug 04 19:11:50.532140 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.532106    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"
Aug 04 19:11:50.532412 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.532164    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" event={"ID":"b214aab8-ad7e-4482-978d-1efc7e68294b","Type":"ContainerDied","Data":"cc3d6627821064e49e2f98f63420bd77f70e8d6ce39ca226b1e34c41ee6e50b3"}
Aug 04 19:11:50.532412 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.532185    2578 scope.go:117] "RemoveContainer" containerID="5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea"
Aug 04 19:11:50.542176 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.542156    2578 scope.go:117] "RemoveContainer" containerID="2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb"
Aug 04 19:11:50.550109 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.550092    2578 scope.go:117] "RemoveContainer" containerID="5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea"
Aug 04 19:11:50.550347 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:11:50.550326    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea\": container with ID starting with 5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea not found: ID does not exist" containerID="5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea"
Aug 04 19:11:50.550436 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.550368    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea"} err="failed to get container status \"5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea\": rpc error: code = NotFound desc = could not find container \"5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea\": container with ID starting with 5820c71af8f17bfc90297efdc1524ec9f2b428fddf9a7b6fcc4ff3b7611e8dea not found: ID does not exist"
Aug 04 19:11:50.550436 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.550393    2578 scope.go:117] "RemoveContainer" containerID="2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb"
Aug 04 19:11:50.550678 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:11:50.550661    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb\": container with ID starting with 2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb not found: ID does not exist" containerID="2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb"
Aug 04 19:11:50.550721 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.550685    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb"} err="failed to get container status \"2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb\": rpc error: code = NotFound desc = could not find container \"2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb\": container with ID starting with 2158da63cc664907ec6f70a273bf7ca878f6dc7b7b4d0771703d348c51fa57bb not found: ID does not exist"
Aug 04 19:11:50.559263 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.559243    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"]
Aug 04 19:11:50.562921 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.562899    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj"]
Aug 04 19:11:50.563618 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.563597    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:50.664070 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.664034    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:11:50.692766 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.692736    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"]
Aug 04 19:11:50.693073 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.693062    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy"
Aug 04 19:11:50.693122 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.693076    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy"
Aug 04 19:11:50.693122 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.693086    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy"
Aug 04 19:11:50.693122 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.693091    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy"
Aug 04 19:11:50.693218 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.693155    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy"
Aug 04 19:11:50.693218 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.693164    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy"
Aug 04 19:11:50.697444 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.697429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:11:50.701459 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.701439    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-c028db-dockercfg-9gfz9\""
Aug 04 19:11:50.701532 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.701439    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-c028db\""
Aug 04 19:11:50.710552 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.710532    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"]
Aug 04 19:11:50.723859 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.723829    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqgbj\" (UniqueName: \"kubernetes.io/projected/22fae719-c22b-46be-9a55-4eb2a1bdc512-kube-api-access-fqgbj\") pod \"payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw\" (UID: \"22fae719-c22b-46be-9a55-4eb2a1bdc512\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:11:50.724020 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.723942    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22fae719-c22b-46be-9a55-4eb2a1bdc512-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw\" (UID: \"22fae719-c22b-46be-9a55-4eb2a1bdc512\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:11:50.802123 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.802091    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" path="/var/lib/kubelet/pods/b214aab8-ad7e-4482-978d-1efc7e68294b/volumes"
Aug 04 19:11:50.824685 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.824650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22fae719-c22b-46be-9a55-4eb2a1bdc512-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw\" (UID: \"22fae719-c22b-46be-9a55-4eb2a1bdc512\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:11:50.824860 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.824726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqgbj\" (UniqueName: \"kubernetes.io/projected/22fae719-c22b-46be-9a55-4eb2a1bdc512-kube-api-access-fqgbj\") pod \"payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw\" (UID: \"22fae719-c22b-46be-9a55-4eb2a1bdc512\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:11:50.825395 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.825376    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22fae719-c22b-46be-9a55-4eb2a1bdc512-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw\" (UID: \"22fae719-c22b-46be-9a55-4eb2a1bdc512\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:11:50.836124 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:50.836090    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqgbj\" (UniqueName: \"kubernetes.io/projected/22fae719-c22b-46be-9a55-4eb2a1bdc512-kube-api-access-fqgbj\") pod \"payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw\" (UID: \"22fae719-c22b-46be-9a55-4eb2a1bdc512\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:11:51.007664 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:51.007629    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:11:51.142563 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:51.142534    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"]
Aug 04 19:11:51.143743 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:11:51.143716    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22fae719_c22b_46be_9a55_4eb2a1bdc512.slice/crio-5e189a37386967050bb05a9f4f00f76be9253e4b7dd3d71562da0829902dfa19 WatchSource:0}: Error finding container 5e189a37386967050bb05a9f4f00f76be9253e4b7dd3d71562da0829902dfa19: Status 404 returned error can't find the container with id 5e189a37386967050bb05a9f4f00f76be9253e4b7dd3d71562da0829902dfa19
Aug 04 19:11:51.235651 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:51.235618    2578 patch_prober.go:28] interesting pod/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Aug 04 19:11:51.235776 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:51.235672    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-77907c-openshift-default-844869fd78-fk7qj" podUID="b214aab8-ad7e-4482-978d-1efc7e68294b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Aug 04 19:11:51.538376 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:51.538342    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw" event={"ID":"22fae719-c22b-46be-9a55-4eb2a1bdc512","Type":"ContainerStarted","Data":"96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30"}
Aug 04 19:11:51.538537 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:51.538496    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:11:51.538700 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:51.538542    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw" event={"ID":"22fae719-c22b-46be-9a55-4eb2a1bdc512","Type":"ContainerStarted","Data":"5e189a37386967050bb05a9f4f00f76be9253e4b7dd3d71562da0829902dfa19"}
Aug 04 19:11:51.558389 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:11:51.558347    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw" podStartSLOduration=1.55833221 podStartE2EDuration="1.55833221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:11:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:11:51.555724839 +0000 UTC m=+2653.422639939" watchObservedRunningTime="2026-08-04 19:11:51.55833221 +0000 UTC m=+2653.425247309"
Aug 04 19:12:02.516850 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:02.516810    2578 patch_prober.go:28] interesting pod/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Aug 04 19:12:02.517253 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:02.516894    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Aug 04 19:12:02.548087 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:02.548049    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw" podUID="22fae719-c22b-46be-9a55-4eb2a1bdc512" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.53:9004: connect: connection refused"
Aug 04 19:12:02.596459 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:02.596427    2578 generic.go:356] "Generic (PLEG): container finished" podID="06185225-8261-47ee-9caf-0064672bc414" containerID="b7a8228b20780d279ac1456e0629e904a65cae209a5212ce55a29ab26a8b501f" exitCode=0
Aug 04 19:12:02.596642 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:02.596501    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" event={"ID":"06185225-8261-47ee-9caf-0064672bc414","Type":"ContainerDied","Data":"b7a8228b20780d279ac1456e0629e904a65cae209a5212ce55a29ab26a8b501f"}
Aug 04 19:12:02.596928 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:02.596913    2578 scope.go:117] "RemoveContainer" containerID="b7a8228b20780d279ac1456e0629e904a65cae209a5212ce55a29ab26a8b501f"
Aug 04 19:12:02.597829 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:02.597802    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:12:02.597878 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:02.597867    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:12:02.597919 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:02.597898    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:12:03.603330 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:03.603298    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" event={"ID":"06185225-8261-47ee-9caf-0064672bc414","Type":"ContainerStarted","Data":"5b20d12810e548bf41427bd751aad237e4a367695b768ef07ba8fa04354aa2cd"}
Aug 04 19:12:04.275228 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:04.275193    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:12:04.275403 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:04.275374    2578 patch_prober.go:28] interesting pod/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Aug 04 19:12:04.275446 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:04.275419    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Aug 04 19:12:05.275713 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:05.275675    2578 patch_prober.go:28] interesting pod/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Aug 04 19:12:05.276128 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:05.275730    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Aug 04 19:12:06.274977 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:06.274943    2578 patch_prober.go:28] interesting pod/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Aug 04 19:12:06.275153 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:06.274997    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Aug 04 19:12:07.275109 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:07.275072    2578 patch_prober.go:28] interesting pod/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Aug 04 19:12:07.275478 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:07.275125    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Aug 04 19:12:08.275784 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:08.275745    2578 patch_prober.go:28] interesting pod/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Aug 04 19:12:08.276171 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:08.275800    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Aug 04 19:12:09.278623 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:09.278593    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:12:09.278988 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:09.278862    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:12:09.279431 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:09.279416    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:12:12.547593 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:12.547538    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:12:18.687794 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:18.687747    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:19.155920 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:19.155875    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:19.470228 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:19.470154    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:19.960805 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:19.960771    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:22.643446 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:22.643407    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:22.767725 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:22.767681    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:23.746451 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:23.746412    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:24.186885 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.186855    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:24.412119 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.412072    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:24.519607 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.519511    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:24.559116 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.559092    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"]
Aug 04 19:12:24.562622 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.562603    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:12:24.565132 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.565111    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-7afb35\""
Aug 04 19:12:24.565244 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.565199    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-7afb35-dockercfg-92h84\""
Aug 04 19:12:24.573856 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.573835    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"]
Aug 04 19:12:24.728898 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.728864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw77n\" (UniqueName: \"kubernetes.io/projected/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-kube-api-access-sw77n\") pod \"payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x\" (UID: \"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9\") " pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:12:24.729067 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.728946    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x\" (UID: \"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9\") " pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:12:24.829928 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.829899    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw77n\" (UniqueName: \"kubernetes.io/projected/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-kube-api-access-sw77n\") pod \"payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x\" (UID: \"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9\") " pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:12:24.830288 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.829965    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x\" (UID: \"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9\") " pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:12:24.830676 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.830658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x\" (UID: \"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9\") " pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:12:24.838471 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.838452    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw77n\" (UniqueName: \"kubernetes.io/projected/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-kube-api-access-sw77n\") pod \"payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x\" (UID: \"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9\") " pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:12:24.872593 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:24.872553    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:12:25.034549 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:25.034527    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"]
Aug 04 19:12:25.036830 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:12:25.036804    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6a4bd50_30eb_4027_bdfc_a646dba5a3b9.slice/crio-97163548ef7e83ab42ab8b8645e0465a5f00d0c4e100fad9573c7ed5b80e8db7 WatchSource:0}: Error finding container 97163548ef7e83ab42ab8b8645e0465a5f00d0c4e100fad9573c7ed5b80e8db7: Status 404 returned error can't find the container with id 97163548ef7e83ab42ab8b8645e0465a5f00d0c4e100fad9573c7ed5b80e8db7
Aug 04 19:12:25.716981 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:25.716947    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x" event={"ID":"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9","Type":"ContainerStarted","Data":"246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3"}
Aug 04 19:12:25.717186 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:25.717018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x" event={"ID":"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9","Type":"ContainerStarted","Data":"97163548ef7e83ab42ab8b8645e0465a5f00d0c4e100fad9573c7ed5b80e8db7"}
Aug 04 19:12:25.717186 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:25.717105    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:12:25.738251 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:25.738206    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x" podStartSLOduration=1.738192684 podStartE2EDuration="1.738192684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:12:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:12:25.73636177 +0000 UTC m=+2687.603276870" watchObservedRunningTime="2026-08-04 19:12:25.738192684 +0000 UTC m=+2687.605107784"
Aug 04 19:12:36.724842 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:36.724764    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:12:37.305615 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:37.305561    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:52.350318 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.350272    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:52.394323 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.394291    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"]
Aug 04 19:12:52.394548 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.394524    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw" podUID="22fae719-c22b-46be-9a55-4eb2a1bdc512" containerName="payload-pre-processing" containerID="cri-o://96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30" gracePeriod=30
Aug 04 19:12:52.505342 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.505314    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:52.547926 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.547888    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw" podUID="22fae719-c22b-46be-9a55-4eb2a1bdc512" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.53:9004: connect: connection refused"
Aug 04 19:12:52.605539 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.605458    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:52.638950 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.638930    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:12:52.775780 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.775752    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqgbj\" (UniqueName: \"kubernetes.io/projected/22fae719-c22b-46be-9a55-4eb2a1bdc512-kube-api-access-fqgbj\") pod \"22fae719-c22b-46be-9a55-4eb2a1bdc512\" (UID: \"22fae719-c22b-46be-9a55-4eb2a1bdc512\") "
Aug 04 19:12:52.775942 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.775863    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22fae719-c22b-46be-9a55-4eb2a1bdc512-plugins-config-volume\") pod \"22fae719-c22b-46be-9a55-4eb2a1bdc512\" (UID: \"22fae719-c22b-46be-9a55-4eb2a1bdc512\") "
Aug 04 19:12:52.776236 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.776213    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22fae719-c22b-46be-9a55-4eb2a1bdc512-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "22fae719-c22b-46be-9a55-4eb2a1bdc512" (UID: "22fae719-c22b-46be-9a55-4eb2a1bdc512"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:12:52.777910 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.777893    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22fae719-c22b-46be-9a55-4eb2a1bdc512-kube-api-access-fqgbj" (OuterVolumeSpecName: "kube-api-access-fqgbj") pod "22fae719-c22b-46be-9a55-4eb2a1bdc512" (UID: "22fae719-c22b-46be-9a55-4eb2a1bdc512"). InnerVolumeSpecName "kube-api-access-fqgbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:12:52.845678 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.845645    2578 generic.go:356] "Generic (PLEG): container finished" podID="22fae719-c22b-46be-9a55-4eb2a1bdc512" containerID="96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30" exitCode=0
Aug 04 19:12:52.845819 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.845691    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"
Aug 04 19:12:52.845819 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.845727    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw" event={"ID":"22fae719-c22b-46be-9a55-4eb2a1bdc512","Type":"ContainerDied","Data":"96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30"}
Aug 04 19:12:52.845819 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.845775    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw" event={"ID":"22fae719-c22b-46be-9a55-4eb2a1bdc512","Type":"ContainerDied","Data":"5e189a37386967050bb05a9f4f00f76be9253e4b7dd3d71562da0829902dfa19"}
Aug 04 19:12:52.845819 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.845798    2578 scope.go:117] "RemoveContainer" containerID="96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30"
Aug 04 19:12:52.853888 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.853861    2578 scope.go:117] "RemoveContainer" containerID="96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30"
Aug 04 19:12:52.854106 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:12:52.854086    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30\": container with ID starting with 96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30 not found: ID does not exist" containerID="96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30"
Aug 04 19:12:52.854152 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.854114    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30"} err="failed to get container status \"96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30\": rpc error: code = NotFound desc = could not find container \"96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30\": container with ID starting with 96d89608433655ddbe41ef99f2782b5487f5a8e169317ac59e56094a2d1fad30 not found: ID does not exist"
Aug 04 19:12:52.864473 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.864425    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"]
Aug 04 19:12:52.868156 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.868136    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c028db-66bc6cff48-8f4xw"]
Aug 04 19:12:52.876813 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.876796    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqgbj\" (UniqueName: \"kubernetes.io/projected/22fae719-c22b-46be-9a55-4eb2a1bdc512-kube-api-access-fqgbj\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:12:52.876870 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:52.876816    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22fae719-c22b-46be-9a55-4eb2a1bdc512-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:12:53.248621 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:53.248515    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:53.350017 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:53.349980    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:54.809055 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:54.809020    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22fae719-c22b-46be-9a55-4eb2a1bdc512" path="/var/lib/kubelet/pods/22fae719-c22b-46be-9a55-4eb2a1bdc512/volumes"
Aug 04 19:12:58.269451 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:58.269413    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:59.457001 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:59.456958    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:59.555626 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:59.555598    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:59.681896 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:59.681859    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"]
Aug 04 19:12:59.682068 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:59.682049    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy" containerID="cri-o://5b20d12810e548bf41427bd751aad237e4a367695b768ef07ba8fa04354aa2cd" gracePeriod=30
Aug 04 19:12:59.704117 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:59.704080    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:12:59.926913 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:12:59.926875    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:04.901882 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:04.901852    2578 generic.go:356] "Generic (PLEG): container finished" podID="06185225-8261-47ee-9caf-0064672bc414" containerID="5b20d12810e548bf41427bd751aad237e4a367695b768ef07ba8fa04354aa2cd" exitCode=0
Aug 04 19:13:04.902297 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:04.901889    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" event={"ID":"06185225-8261-47ee-9caf-0064672bc414","Type":"ContainerDied","Data":"5b20d12810e548bf41427bd751aad237e4a367695b768ef07ba8fa04354aa2cd"}
Aug 04 19:13:04.902297 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:04.901946    2578 scope.go:117] "RemoveContainer" containerID="b7a8228b20780d279ac1456e0629e904a65cae209a5212ce55a29ab26a8b501f"
Aug 04 19:13:04.934223 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:04.934198    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:13:05.086040 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086011    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-istio-token\") pod \"06185225-8261-47ee-9caf-0064672bc414\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") "
Aug 04 19:13:05.086040 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086041    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxnv4\" (UniqueName: \"kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-kube-api-access-fxnv4\") pod \"06185225-8261-47ee-9caf-0064672bc414\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") "
Aug 04 19:13:05.086272 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086107    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-envoy\") pod \"06185225-8261-47ee-9caf-0064672bc414\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") "
Aug 04 19:13:05.086272 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086125    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-socket\") pod \"06185225-8261-47ee-9caf-0064672bc414\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") "
Aug 04 19:13:05.086272 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086151    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-credential-socket\") pod \"06185225-8261-47ee-9caf-0064672bc414\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") "
Aug 04 19:13:05.086272 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086172    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-data\") pod \"06185225-8261-47ee-9caf-0064672bc414\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") "
Aug 04 19:13:05.086272 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086199    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06185225-8261-47ee-9caf-0064672bc414-istiod-ca-cert\") pod \"06185225-8261-47ee-9caf-0064672bc414\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") "
Aug 04 19:13:05.086272 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086244    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-certs\") pod \"06185225-8261-47ee-9caf-0064672bc414\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") "
Aug 04 19:13:05.086272 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086269    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06185225-8261-47ee-9caf-0064672bc414-istio-podinfo\") pod \"06185225-8261-47ee-9caf-0064672bc414\" (UID: \"06185225-8261-47ee-9caf-0064672bc414\") "
Aug 04 19:13:05.086625 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086531    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "06185225-8261-47ee-9caf-0064672bc414" (UID: "06185225-8261-47ee-9caf-0064672bc414"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:13:05.086691 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086656    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-data" (OuterVolumeSpecName: "istio-data") pod "06185225-8261-47ee-9caf-0064672bc414" (UID: "06185225-8261-47ee-9caf-0064672bc414"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:13:05.086748 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086688    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "06185225-8261-47ee-9caf-0064672bc414" (UID: "06185225-8261-47ee-9caf-0064672bc414"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:13:05.086799 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086759    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06185225-8261-47ee-9caf-0064672bc414-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "06185225-8261-47ee-9caf-0064672bc414" (UID: "06185225-8261-47ee-9caf-0064672bc414"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:13:05.086880 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.086817    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "06185225-8261-47ee-9caf-0064672bc414" (UID: "06185225-8261-47ee-9caf-0064672bc414"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:13:05.088443 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.088414    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-kube-api-access-fxnv4" (OuterVolumeSpecName: "kube-api-access-fxnv4") pod "06185225-8261-47ee-9caf-0064672bc414" (UID: "06185225-8261-47ee-9caf-0064672bc414"). InnerVolumeSpecName "kube-api-access-fxnv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:13:05.088551 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.088443    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06185225-8261-47ee-9caf-0064672bc414-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "06185225-8261-47ee-9caf-0064672bc414" (UID: "06185225-8261-47ee-9caf-0064672bc414"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 19:13:05.088689 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.088671    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "06185225-8261-47ee-9caf-0064672bc414" (UID: "06185225-8261-47ee-9caf-0064672bc414"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:13:05.088739 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.088699    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-istio-token" (OuterVolumeSpecName: "istio-token") pod "06185225-8261-47ee-9caf-0064672bc414" (UID: "06185225-8261-47ee-9caf-0064672bc414"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:13:05.187114 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.187087    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06185225-8261-47ee-9caf-0064672bc414-istiod-ca-cert\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:13:05.187114 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.187109    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:13:05.187114 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.187118    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06185225-8261-47ee-9caf-0064672bc414-istio-podinfo\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:13:05.187318 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.187126    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-istio-token\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:13:05.187318 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.187136    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxnv4\" (UniqueName: \"kubernetes.io/projected/06185225-8261-47ee-9caf-0064672bc414-kube-api-access-fxnv4\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:13:05.187318 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.187147    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-envoy\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:13:05.187318 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.187155    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-workload-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:13:05.187318 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.187164    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-credential-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:13:05.187318 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.187171    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06185225-8261-47ee-9caf-0064672bc414-istio-data\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:13:05.913045 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.913015    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"
Aug 04 19:13:05.913497 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.913042    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95" event={"ID":"06185225-8261-47ee-9caf-0064672bc414","Type":"ContainerDied","Data":"c284b404f6d2a29d5d79b4e9fffea6e03f6913a3cf392c93443353950d48895a"}
Aug 04 19:13:05.913497 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.913116    2578 scope.go:117] "RemoveContainer" containerID="5b20d12810e548bf41427bd751aad237e4a367695b768ef07ba8fa04354aa2cd"
Aug 04 19:13:05.938047 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.938024    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"]
Aug 04 19:13:05.944266 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:05.944242    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c028db-openshift-default-5479dcb646-t2j95"]
Aug 04 19:13:06.801551 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:06.801519    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06185225-8261-47ee-9caf-0064672bc414" path="/var/lib/kubelet/pods/06185225-8261-47ee-9caf-0064672bc414/volumes"
Aug 04 19:13:10.245642 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.245570    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:10.268303 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.268277    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"]
Aug 04 19:13:10.268540 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.268515    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x" podUID="e6a4bd50-30eb-4027-bdfc-a646dba5a3b9" containerName="payload-processing" containerID="cri-o://246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3" gracePeriod=30
Aug 04 19:13:10.522889 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.522865    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:13:10.632772 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.632747    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-plugins-config-volume\") pod \"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9\" (UID: \"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9\") "
Aug 04 19:13:10.632914 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.632810    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw77n\" (UniqueName: \"kubernetes.io/projected/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-kube-api-access-sw77n\") pod \"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9\" (UID: \"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9\") "
Aug 04 19:13:10.633171 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.633148    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6a4bd50-30eb-4027-bdfc-a646dba5a3b9" (UID: "e6a4bd50-30eb-4027-bdfc-a646dba5a3b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:13:10.634817 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.634794    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-kube-api-access-sw77n" (OuterVolumeSpecName: "kube-api-access-sw77n") pod "e6a4bd50-30eb-4027-bdfc-a646dba5a3b9" (UID: "e6a4bd50-30eb-4027-bdfc-a646dba5a3b9"). InnerVolumeSpecName "kube-api-access-sw77n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:13:10.733916 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.733880    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:13:10.733916 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.733910    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw77n\" (UniqueName: \"kubernetes.io/projected/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9-kube-api-access-sw77n\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:13:10.938165 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.938124    2578 generic.go:356] "Generic (PLEG): container finished" podID="e6a4bd50-30eb-4027-bdfc-a646dba5a3b9" containerID="246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3" exitCode=0
Aug 04 19:13:10.938307 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.938170    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x" event={"ID":"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9","Type":"ContainerDied","Data":"246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3"}
Aug 04 19:13:10.938307 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.938180    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"
Aug 04 19:13:10.938307 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.938219    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x" event={"ID":"e6a4bd50-30eb-4027-bdfc-a646dba5a3b9","Type":"ContainerDied","Data":"97163548ef7e83ab42ab8b8645e0465a5f00d0c4e100fad9573c7ed5b80e8db7"}
Aug 04 19:13:10.938307 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.938241    2578 scope.go:117] "RemoveContainer" containerID="246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3"
Aug 04 19:13:10.946158 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.946140    2578 scope.go:117] "RemoveContainer" containerID="246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3"
Aug 04 19:13:10.946397 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:13:10.946378    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3\": container with ID starting with 246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3 not found: ID does not exist" containerID="246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3"
Aug 04 19:13:10.946465 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.946410    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3"} err="failed to get container status \"246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3\": rpc error: code = NotFound desc = could not find container \"246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3\": container with ID starting with 246b98f517471ac002f8eba3695fe1c2db729db8555801b3957d925b6bdb74e3 not found: ID does not exist"
Aug 04 19:13:10.954874 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.954854    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"]
Aug 04 19:13:10.959084 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:10.959065    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7afb35-5cdbf6dc54-w2t7x"]
Aug 04 19:13:12.455731 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:12.455697    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:12.803088 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:12.803060    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6a4bd50-30eb-4027-bdfc-a646dba5a3b9" path="/var/lib/kubelet/pods/e6a4bd50-30eb-4027-bdfc-a646dba5a3b9/volumes"
Aug 04 19:13:16.831828 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:16.831798    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:16.903716 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:16.903682    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:17.118549 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.118464    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:17.339567 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.339536    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:17.678242 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.678201    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:17.955012 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.954922    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"]
Aug 04 19:13:17.955397 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.955277    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6a4bd50-30eb-4027-bdfc-a646dba5a3b9" containerName="payload-processing"
Aug 04 19:13:17.955397 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.955290    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6a4bd50-30eb-4027-bdfc-a646dba5a3b9" containerName="payload-processing"
Aug 04 19:13:17.955397 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.955299    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy"
Aug 04 19:13:17.955397 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.955305    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy"
Aug 04 19:13:17.955397 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.955313    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22fae719-c22b-46be-9a55-4eb2a1bdc512" containerName="payload-pre-processing"
Aug 04 19:13:17.955397 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.955319    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="22fae719-c22b-46be-9a55-4eb2a1bdc512" containerName="payload-pre-processing"
Aug 04 19:13:17.955761 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.955427    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy"
Aug 04 19:13:17.955761 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.955446    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy"
Aug 04 19:13:17.955761 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.955458    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="22fae719-c22b-46be-9a55-4eb2a1bdc512" containerName="payload-pre-processing"
Aug 04 19:13:17.955761 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.955470    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6a4bd50-30eb-4027-bdfc-a646dba5a3b9" containerName="payload-processing"
Aug 04 19:13:17.958416 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.958402    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:17.961060 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.961043    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-1d2c46-openshift-default-dockercfg-qjm8j\""
Aug 04 19:13:17.971165 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.971145    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"]
Aug 04 19:13:17.982173 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.982150    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:17.996375 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.996352    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldcgc\" (UniqueName: \"kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-kube-api-access-ldcgc\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:17.996472 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.996379    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-token\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:17.996472 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.996415    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istiod-ca-cert\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:17.996472 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.996460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-socket\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:17.996599 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.996479    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-data\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:17.996599 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.996496    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-podinfo\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:17.996599 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.996523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-credential-socket\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:17.996599 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.996538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-certs\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:17.996599 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:17.996555    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-envoy\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097110 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097075    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-socket\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097110 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-data\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097344 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097128    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-podinfo\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097344 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097163    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-credential-socket\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097344 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097178    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-certs\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097344 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097196    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-envoy\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097344 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097229    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldcgc\" (UniqueName: \"kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-kube-api-access-ldcgc\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097344 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-token\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097344 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istiod-ca-cert\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097704 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097543    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-data\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097704 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097600    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-credential-socket\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097704 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-certs\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.097888 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097871    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-socket\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.098013 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.097996    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istiod-ca-cert\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.099478 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.099458    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-envoy\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.099631 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.099612    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-podinfo\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.105357 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.105335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldcgc\" (UniqueName: \"kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-kube-api-access-ldcgc\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.105812 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.105790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-token\") pod \"e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") " pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.268525 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.268446    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:18.360939 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.360906    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:18.408456 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.408426    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"]
Aug 04 19:13:18.411330 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:13:18.411302    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed58469_64e8_4a86_b99e_7096e2e5f42b.slice/crio-74316bfeeb47c3456cf108fb821d32143c0ed7e5a2acef23fa6e6c287073059d WatchSource:0}: Error finding container 74316bfeeb47c3456cf108fb821d32143c0ed7e5a2acef23fa6e6c287073059d: Status 404 returned error can't find the container with id 74316bfeeb47c3456cf108fb821d32143c0ed7e5a2acef23fa6e6c287073059d
Aug 04 19:13:18.413795 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.413750    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:13:18.413901 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.413840    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:13:18.413901 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.413882    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:13:18.977963 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.977925    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" event={"ID":"9ed58469-64e8-4a86-b99e-7096e2e5f42b","Type":"ContainerStarted","Data":"0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a"}
Aug 04 19:13:18.978349 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.977989    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" event={"ID":"9ed58469-64e8-4a86-b99e-7096e2e5f42b","Type":"ContainerStarted","Data":"74316bfeeb47c3456cf108fb821d32143c0ed7e5a2acef23fa6e6c287073059d"}
Aug 04 19:13:18.999864 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:18.999819    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" podStartSLOduration=1.999806741 podStartE2EDuration="1.999806741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:13:18.998468098 +0000 UTC m=+2740.865383211" watchObservedRunningTime="2026-08-04 19:13:18.999806741 +0000 UTC m=+2740.866721840"
Aug 04 19:13:19.269164 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:19.269084    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:19.274637 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:19.274617    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:19.983646 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:19.983617    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:19.984609 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:19.984564    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:20.906201 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:20.906168    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:21.007205 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:21.007174    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:22.039315 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.039277    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:22.539350 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.539312    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:22.635665 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.635633    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:22.636598 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.636556    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"]
Aug 04 19:13:22.636964 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.636948    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy"
Aug 04 19:13:22.637033 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.636967    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="06185225-8261-47ee-9caf-0064672bc414" containerName="istio-proxy"
Aug 04 19:13:22.640130 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.640111    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:13:22.642666 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.642647    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-1d2c46\""
Aug 04 19:13:22.651380 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.651358    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"]
Aug 04 19:13:22.740269 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.740233    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfb1071-9804-4a80-a66e-489a61bc6938-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp\" (UID: \"ecfb1071-9804-4a80-a66e-489a61bc6938\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:13:22.740269 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.740266    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc9vn\" (UniqueName: \"kubernetes.io/projected/ecfb1071-9804-4a80-a66e-489a61bc6938-kube-api-access-cc9vn\") pod \"payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp\" (UID: \"ecfb1071-9804-4a80-a66e-489a61bc6938\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:13:22.754369 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.754340    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:22.841171 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.841085    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfb1071-9804-4a80-a66e-489a61bc6938-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp\" (UID: \"ecfb1071-9804-4a80-a66e-489a61bc6938\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:13:22.841171 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.841118    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc9vn\" (UniqueName: \"kubernetes.io/projected/ecfb1071-9804-4a80-a66e-489a61bc6938-kube-api-access-cc9vn\") pod \"payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp\" (UID: \"ecfb1071-9804-4a80-a66e-489a61bc6938\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:13:22.841808 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.841790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfb1071-9804-4a80-a66e-489a61bc6938-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp\" (UID: \"ecfb1071-9804-4a80-a66e-489a61bc6938\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:13:22.850060 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.850041    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc9vn\" (UniqueName: \"kubernetes.io/projected/ecfb1071-9804-4a80-a66e-489a61bc6938-kube-api-access-cc9vn\") pod \"payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp\" (UID: \"ecfb1071-9804-4a80-a66e-489a61bc6938\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:13:22.950705 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:22.950673    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:13:23.084748 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:23.084725    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"]
Aug 04 19:13:23.086511 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:13:23.086484    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecfb1071_9804_4a80_a66e_489a61bc6938.slice/crio-5eb53540b0cdfe3dd5bdb7f560ad807ae606d38ad0b5aea7da8e964e222b1b58 WatchSource:0}: Error finding container 5eb53540b0cdfe3dd5bdb7f560ad807ae606d38ad0b5aea7da8e964e222b1b58: Status 404 returned error can't find the container with id 5eb53540b0cdfe3dd5bdb7f560ad807ae606d38ad0b5aea7da8e964e222b1b58
Aug 04 19:13:23.178428 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:23.178402    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:24.014052 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:24.014017    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp" event={"ID":"ecfb1071-9804-4a80-a66e-489a61bc6938","Type":"ContainerStarted","Data":"1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae"}
Aug 04 19:13:24.014248 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:24.014071    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp" event={"ID":"ecfb1071-9804-4a80-a66e-489a61bc6938","Type":"ContainerStarted","Data":"5eb53540b0cdfe3dd5bdb7f560ad807ae606d38ad0b5aea7da8e964e222b1b58"}
Aug 04 19:13:24.014248 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:24.014166    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:13:24.032747 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:24.032706    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp" podStartSLOduration=2.032692877 podStartE2EDuration="2.032692877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:13:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:13:24.030164467 +0000 UTC m=+2745.897079568" watchObservedRunningTime="2026-08-04 19:13:24.032692877 +0000 UTC m=+2745.899607976"
Aug 04 19:13:34.071350 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:34.071317    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerID="0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a" exitCode=0
Aug 04 19:13:34.071829 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:34.071357    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" event={"ID":"9ed58469-64e8-4a86-b99e-7096e2e5f42b","Type":"ContainerDied","Data":"0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a"}
Aug 04 19:13:34.071829 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:34.071691    2578 scope.go:117] "RemoveContainer" containerID="0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a"
Aug 04 19:13:34.072614 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:34.072587    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:13:34.072717 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:34.072648    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:13:34.072717 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:34.072678    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:13:34.984500 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:34.984468    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:35.023661 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:35.023628    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:13:35.078708 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:35.078678    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" event={"ID":"9ed58469-64e8-4a86-b99e-7096e2e5f42b","Type":"ContainerStarted","Data":"f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613"}
Aug 04 19:13:35.268649 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:35.268544    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:35.269784 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:35.269759    2578 patch_prober.go:28] interesting pod/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Aug 04 19:13:35.269885 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:35.269820    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Aug 04 19:13:36.269428 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:36.269396    2578 patch_prober.go:28] interesting pod/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Aug 04 19:13:36.269816 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:36.269443    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Aug 04 19:13:37.269223 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:37.269188    2578 patch_prober.go:28] interesting pod/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Aug 04 19:13:37.269427 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:37.269238    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Aug 04 19:13:38.269156 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:38.269116    2578 patch_prober.go:28] interesting pod/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Aug 04 19:13:38.269537 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:38.269171    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Aug 04 19:13:39.269366 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:39.269282    2578 patch_prober.go:28] interesting pod/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Aug 04 19:13:39.269366 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:39.269345    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Aug 04 19:13:40.272804 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:40.272775    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:40.273248 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:40.273129    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:40.273815 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:40.273797    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:13:50.979703 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:50.979671    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:51.322797 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:51.322767    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:53.057124 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:53.057082    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:53.157663 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:53.157626    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:54.135941 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:54.135907    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:54.229445 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:54.229411    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:55.256637 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.256600    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:55.356442 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.356407    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:55.456560 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.456522    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:55.556291 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.556254    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:55.866153 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.866065    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:55.962635 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.962605    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"]
Aug 04 19:13:55.964910 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.964868    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:13:55.966161 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.966142    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:13:55.968858 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.968841    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-1a00f6\""
Aug 04 19:13:55.968949 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.968864    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-1a00f6-dockercfg-wnljf\""
Aug 04 19:13:55.975990 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.975968    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"]
Aug 04 19:13:55.979763 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.979744    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"]
Aug 04 19:13:55.982956 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.982942    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:13:55.995078 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:55.995059    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"]
Aug 04 19:13:56.135719 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.135636    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38682654-a0a1-432d-b404-6f97bd365452-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd\" (UID: \"38682654-a0a1-432d-b404-6f97bd365452\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:13:56.135868 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.135723    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jljgn\" (UniqueName: \"kubernetes.io/projected/38682654-a0a1-432d-b404-6f97bd365452-kube-api-access-jljgn\") pod \"payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd\" (UID: \"38682654-a0a1-432d-b404-6f97bd365452\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:13:56.135868 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.135821    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b02c38d2-440f-4b44-ae0c-151f12158b77-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm\" (UID: \"b02c38d2-440f-4b44-ae0c-151f12158b77\") " pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:13:56.135868 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.135845    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64zpm\" (UniqueName: \"kubernetes.io/projected/b02c38d2-440f-4b44-ae0c-151f12158b77-kube-api-access-64zpm\") pod \"payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm\" (UID: \"b02c38d2-440f-4b44-ae0c-151f12158b77\") " pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:13:56.236927 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.236895    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jljgn\" (UniqueName: \"kubernetes.io/projected/38682654-a0a1-432d-b404-6f97bd365452-kube-api-access-jljgn\") pod \"payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd\" (UID: \"38682654-a0a1-432d-b404-6f97bd365452\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:13:56.237099 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.236985    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b02c38d2-440f-4b44-ae0c-151f12158b77-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm\" (UID: \"b02c38d2-440f-4b44-ae0c-151f12158b77\") " pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:13:56.237099 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.237012    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64zpm\" (UniqueName: \"kubernetes.io/projected/b02c38d2-440f-4b44-ae0c-151f12158b77-kube-api-access-64zpm\") pod \"payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm\" (UID: \"b02c38d2-440f-4b44-ae0c-151f12158b77\") " pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:13:56.237099 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.237078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38682654-a0a1-432d-b404-6f97bd365452-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd\" (UID: \"38682654-a0a1-432d-b404-6f97bd365452\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:13:56.237761 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.237737    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38682654-a0a1-432d-b404-6f97bd365452-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd\" (UID: \"38682654-a0a1-432d-b404-6f97bd365452\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:13:56.237903 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.237762    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b02c38d2-440f-4b44-ae0c-151f12158b77-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm\" (UID: \"b02c38d2-440f-4b44-ae0c-151f12158b77\") " pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:13:56.245910 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.245886    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64zpm\" (UniqueName: \"kubernetes.io/projected/b02c38d2-440f-4b44-ae0c-151f12158b77-kube-api-access-64zpm\") pod \"payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm\" (UID: \"b02c38d2-440f-4b44-ae0c-151f12158b77\") " pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:13:56.246005 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.245923    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jljgn\" (UniqueName: \"kubernetes.io/projected/38682654-a0a1-432d-b404-6f97bd365452-kube-api-access-jljgn\") pod \"payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd\" (UID: \"38682654-a0a1-432d-b404-6f97bd365452\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:13:56.276917 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.276893    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:13:56.293000 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.292979    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:13:56.463184 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.463162    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"]
Aug 04 19:13:56.464778 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:13:56.464751    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02c38d2_440f_4b44_ae0c_151f12158b77.slice/crio-f4ad329923c11f89170a57a96d4af702413f5ca612727ed8f7987f753d7152fb WatchSource:0}: Error finding container f4ad329923c11f89170a57a96d4af702413f5ca612727ed8f7987f753d7152fb: Status 404 returned error can't find the container with id f4ad329923c11f89170a57a96d4af702413f5ca612727ed8f7987f753d7152fb
Aug 04 19:13:56.642824 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:56.642800    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"]
Aug 04 19:13:56.644704 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:13:56.644677    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38682654_a0a1_432d_b404_6f97bd365452.slice/crio-606918a859bc27dd0cccbebafeb74191900af789818c351bc5cf14b48e9a046a WatchSource:0}: Error finding container 606918a859bc27dd0cccbebafeb74191900af789818c351bc5cf14b48e9a046a: Status 404 returned error can't find the container with id 606918a859bc27dd0cccbebafeb74191900af789818c351bc5cf14b48e9a046a
Aug 04 19:13:57.179362 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:57.179311    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm" event={"ID":"b02c38d2-440f-4b44-ae0c-151f12158b77","Type":"ContainerStarted","Data":"7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681"}
Aug 04 19:13:57.179565 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:57.179385    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm" event={"ID":"b02c38d2-440f-4b44-ae0c-151f12158b77","Type":"ContainerStarted","Data":"f4ad329923c11f89170a57a96d4af702413f5ca612727ed8f7987f753d7152fb"}
Aug 04 19:13:57.179565 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:57.179414    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:13:57.180857 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:57.180826    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd" event={"ID":"38682654-a0a1-432d-b404-6f97bd365452","Type":"ContainerStarted","Data":"8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269"}
Aug 04 19:13:57.180964 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:57.180866    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd" event={"ID":"38682654-a0a1-432d-b404-6f97bd365452","Type":"ContainerStarted","Data":"606918a859bc27dd0cccbebafeb74191900af789818c351bc5cf14b48e9a046a"}
Aug 04 19:13:57.180964 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:57.180899    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:13:57.195209 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:57.195162    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm" podStartSLOduration=2.195144902 podStartE2EDuration="2.195144902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:13:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:13:57.194826583 +0000 UTC m=+2779.061741683" watchObservedRunningTime="2026-08-04 19:13:57.195144902 +0000 UTC m=+2779.062060002"
Aug 04 19:13:57.211187 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:13:57.211150    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd" podStartSLOduration=2.211138476 podStartE2EDuration="2.211138476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:13:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:13:57.209592073 +0000 UTC m=+2779.076507164" watchObservedRunningTime="2026-08-04 19:13:57.211138476 +0000 UTC m=+2779.078053576"
Aug 04 19:14:05.243429 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:05.243344    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:14:05.354481 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:05.354443    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:14:08.187122 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:08.187091    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:14:08.187546 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:08.187160    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:14:15.087462 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:15.087429    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:14:19.115560 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:19.115529    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:14:23.871743 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.871710    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"]
Aug 04 19:14:23.874939 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.874923    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:23.878674 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.878652    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1d2c46\"/\"kube-root-ca.crt\""
Aug 04 19:14:23.878795 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.878677    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1d2c46\"/\"default-dockercfg-5fxxj\""
Aug 04 19:14:23.878795 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.878682    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1d2c46\"/\"auth-test-model-1d2c46-kserve-self-signed-certs\""
Aug 04 19:14:23.878795 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.878744    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1d2c46\"/\"openshift-service-ca.crt\""
Aug 04 19:14:23.884939 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.884917    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"]
Aug 04 19:14:23.970523 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.970489    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:14:23.986598 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.986546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-home\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:23.986722 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.986604    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-tmp-dir\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:23.986722 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.986668    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-model-cache\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:23.986817 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.986735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc9f417-374a-44b0-89b2-0acc59f00218-tls-certs\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:23.986817 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.986754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt9pj\" (UniqueName: \"kubernetes.io/projected/4fc9f417-374a-44b0-89b2-0acc59f00218-kube-api-access-rt9pj\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:23.986817 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:23.986809    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-dshm\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.087968 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.087933    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-home\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.087968 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.087980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-tmp-dir\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.088192 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.088007    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-model-cache\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.088192 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.088058    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc9f417-374a-44b0-89b2-0acc59f00218-tls-certs\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.088192 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.088082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt9pj\" (UniqueName: \"kubernetes.io/projected/4fc9f417-374a-44b0-89b2-0acc59f00218-kube-api-access-rt9pj\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.088192 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.088136    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-dshm\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.088834 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.088809    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-model-cache\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.088984 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.088854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-tmp-dir\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.089096 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.089052    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-home\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.090875 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.090854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-dshm\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.091006 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.090983    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc9f417-374a-44b0-89b2-0acc59f00218-tls-certs\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.097851 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.097807    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt9pj\" (UniqueName: \"kubernetes.io/projected/4fc9f417-374a-44b0-89b2-0acc59f00218-kube-api-access-rt9pj\") pod \"auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") " pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.184874 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.184816    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:24.310840 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:24.310816    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"]
Aug 04 19:14:24.313794 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:14:24.313765    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fc9f417_374a_44b0_89b2_0acc59f00218.slice/crio-a5ccdc857a086dda9247614512509ee5c56621d547518b1e792a1d9eb8532556 WatchSource:0}: Error finding container a5ccdc857a086dda9247614512509ee5c56621d547518b1e792a1d9eb8532556: Status 404 returned error can't find the container with id a5ccdc857a086dda9247614512509ee5c56621d547518b1e792a1d9eb8532556
Aug 04 19:14:25.311344 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:25.311308    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6" event={"ID":"4fc9f417-374a-44b0-89b2-0acc59f00218","Type":"ContainerStarted","Data":"87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125"}
Aug 04 19:14:25.311771 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:25.311353    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6" event={"ID":"4fc9f417-374a-44b0-89b2-0acc59f00218","Type":"ContainerStarted","Data":"a5ccdc857a086dda9247614512509ee5c56621d547518b1e792a1d9eb8532556"}
Aug 04 19:14:25.328535 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:25.328487    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6" podStartSLOduration=2.328472646 podStartE2EDuration="2.328472646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:14:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:14:25.327249291 +0000 UTC m=+2807.194164390" watchObservedRunningTime="2026-08-04 19:14:25.328472646 +0000 UTC m=+2807.195387745"
Aug 04 19:14:34.185499 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:34.185467    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:34.185887 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:34.185530    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:34.197534 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:34.197507    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:14:34.363873 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:14:34.363839    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:15:13.527684 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:15:13.527636    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:17:02.993961 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:17:02.993888    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:18:07.594208 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:07.594172    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"]
Aug 04 19:18:07.618249 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:07.618222    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:18:09.104176 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:09.104133    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:18:14.189228 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:14.189196    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:18:22.260524 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.260488    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:18:22.362202 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.362171    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:18:22.612711 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.612667    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6" podUID="4fc9f417-374a-44b0-89b2-0acc59f00218" containerName="main" containerID="cri-o://87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125" gracePeriod=45
Aug 04 19:18:22.859831 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.859806    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:18:22.930728 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.930653    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-tmp-dir\") pod \"4fc9f417-374a-44b0-89b2-0acc59f00218\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") "
Aug 04 19:18:22.930728 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.930687    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc9f417-374a-44b0-89b2-0acc59f00218-tls-certs\") pod \"4fc9f417-374a-44b0-89b2-0acc59f00218\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") "
Aug 04 19:18:22.930728 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.930706    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-dshm\") pod \"4fc9f417-374a-44b0-89b2-0acc59f00218\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") "
Aug 04 19:18:22.930971 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.930841    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-home\") pod \"4fc9f417-374a-44b0-89b2-0acc59f00218\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") "
Aug 04 19:18:22.930971 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.930903    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-model-cache\") pod \"4fc9f417-374a-44b0-89b2-0acc59f00218\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") "
Aug 04 19:18:22.930971 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.930942    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt9pj\" (UniqueName: \"kubernetes.io/projected/4fc9f417-374a-44b0-89b2-0acc59f00218-kube-api-access-rt9pj\") pod \"4fc9f417-374a-44b0-89b2-0acc59f00218\" (UID: \"4fc9f417-374a-44b0-89b2-0acc59f00218\") "
Aug 04 19:18:22.931134 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.930958    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4fc9f417-374a-44b0-89b2-0acc59f00218" (UID: "4fc9f417-374a-44b0-89b2-0acc59f00218"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:18:22.931189 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.931112    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-home" (OuterVolumeSpecName: "home") pod "4fc9f417-374a-44b0-89b2-0acc59f00218" (UID: "4fc9f417-374a-44b0-89b2-0acc59f00218"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:18:22.931249 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.931184    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-model-cache" (OuterVolumeSpecName: "model-cache") pod "4fc9f417-374a-44b0-89b2-0acc59f00218" (UID: "4fc9f417-374a-44b0-89b2-0acc59f00218"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:18:22.931304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.931256    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-model-cache\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:18:22.931304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.931277    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-tmp-dir\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:18:22.931304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.931290    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-home\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:18:22.932873 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.932849    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-dshm" (OuterVolumeSpecName: "dshm") pod "4fc9f417-374a-44b0-89b2-0acc59f00218" (UID: "4fc9f417-374a-44b0-89b2-0acc59f00218"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:18:22.932873 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.932853    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fc9f417-374a-44b0-89b2-0acc59f00218-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4fc9f417-374a-44b0-89b2-0acc59f00218" (UID: "4fc9f417-374a-44b0-89b2-0acc59f00218"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 19:18:22.933015 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:22.932942    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fc9f417-374a-44b0-89b2-0acc59f00218-kube-api-access-rt9pj" (OuterVolumeSpecName: "kube-api-access-rt9pj") pod "4fc9f417-374a-44b0-89b2-0acc59f00218" (UID: "4fc9f417-374a-44b0-89b2-0acc59f00218"). InnerVolumeSpecName "kube-api-access-rt9pj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:18:23.032286 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.032255    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt9pj\" (UniqueName: \"kubernetes.io/projected/4fc9f417-374a-44b0-89b2-0acc59f00218-kube-api-access-rt9pj\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:18:23.032286 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.032282    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc9f417-374a-44b0-89b2-0acc59f00218-tls-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:18:23.032286 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.032292    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fc9f417-374a-44b0-89b2-0acc59f00218-dshm\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:18:23.435555 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.435520    2578 generic.go:356] "Generic (PLEG): container finished" podID="4fc9f417-374a-44b0-89b2-0acc59f00218" containerID="87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125" exitCode=0
Aug 04 19:18:23.435969 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.435594    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"
Aug 04 19:18:23.435969 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.435624    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6" event={"ID":"4fc9f417-374a-44b0-89b2-0acc59f00218","Type":"ContainerDied","Data":"87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125"}
Aug 04 19:18:23.435969 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.435673    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6" event={"ID":"4fc9f417-374a-44b0-89b2-0acc59f00218","Type":"ContainerDied","Data":"a5ccdc857a086dda9247614512509ee5c56621d547518b1e792a1d9eb8532556"}
Aug 04 19:18:23.435969 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.435691    2578 scope.go:117] "RemoveContainer" containerID="87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125"
Aug 04 19:18:23.444281 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.444264    2578 scope.go:117] "RemoveContainer" containerID="87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125"
Aug 04 19:18:23.444548 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:18:23.444531    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125\": container with ID starting with 87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125 not found: ID does not exist" containerID="87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125"
Aug 04 19:18:23.444669 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.444556    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125"} err="failed to get container status \"87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125\": rpc error: code = NotFound desc = could not find container \"87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125\": container with ID starting with 87614229ec24f3435b21acb957cb07f0b5e6fce9d51299767f1cbc16a26ad125 not found: ID does not exist"
Aug 04 19:18:23.458496 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.458472    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"]
Aug 04 19:18:23.462355 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:23.462333    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1d2c46/auth-test-model-1d2c46-kserve-774f4d58f5-ckfd6"]
Aug 04 19:18:24.802523 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:24.802488    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fc9f417-374a-44b0-89b2-0acc59f00218" path="/var/lib/kubelet/pods/4fc9f417-374a-44b0-89b2-0acc59f00218/volumes"
Aug 04 19:18:35.479572 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.479539    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"]
Aug 04 19:18:35.481947 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.479915    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fc9f417-374a-44b0-89b2-0acc59f00218" containerName="main"
Aug 04 19:18:35.481947 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.479926    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fc9f417-374a-44b0-89b2-0acc59f00218" containerName="main"
Aug 04 19:18:35.481947 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.479982    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fc9f417-374a-44b0-89b2-0acc59f00218" containerName="main"
Aug 04 19:18:35.482875 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.482859    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.486647 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.486628    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1a00f6\"/\"sub-test-model-1a00f6-kserve-self-signed-certs\""
Aug 04 19:18:35.486778 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.486668    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1a00f6\"/\"kube-root-ca.crt\""
Aug 04 19:18:35.486778 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.486667    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1a00f6\"/\"default-dockercfg-7snqf\""
Aug 04 19:18:35.486778 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.486738    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1a00f6\"/\"openshift-service-ca.crt\""
Aug 04 19:18:35.493993 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.493971    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"]
Aug 04 19:18:35.505086 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.505061    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:18:35.640336 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.640295    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tls-certs\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.640336 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.640330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-home\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.640554 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.640346    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-dshm\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.640554 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.640379    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tmp-dir\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.640554 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.640470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tbwz\" (UniqueName: \"kubernetes.io/projected/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-kube-api-access-8tbwz\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.640554 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.640497    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-model-cache\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.741913 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.741838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tbwz\" (UniqueName: \"kubernetes.io/projected/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-kube-api-access-8tbwz\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.741913 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.741870    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-model-cache\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.742132 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.741938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tls-certs\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.742132 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.741956    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-home\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.742132 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.741969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-dshm\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.742132 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.741988    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tmp-dir\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.742342 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.742328    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-model-cache\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.742400 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.742340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tmp-dir\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.742457 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.742443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-home\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.744261 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.744244    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-dshm\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.744520 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.744503    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tls-certs\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.750506 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.750489    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tbwz\" (UniqueName: \"kubernetes.io/projected/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-kube-api-access-8tbwz\") pod \"sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") " pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.792058 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.792034    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:35.921586 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:35.921554    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"]
Aug 04 19:18:35.924197 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:18:35.924170    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9eae0c97_a0ca_4918_9f72_c0e2d97502e9.slice/crio-82c4371c807e468428ea61e9218c018277fdb1fa4bb3faa2396d4822d45ce638 WatchSource:0}: Error finding container 82c4371c807e468428ea61e9218c018277fdb1fa4bb3faa2396d4822d45ce638: Status 404 returned error can't find the container with id 82c4371c807e468428ea61e9218c018277fdb1fa4bb3faa2396d4822d45ce638
Aug 04 19:18:36.498057 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:36.498026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f" event={"ID":"9eae0c97-a0ca-4918-9f72-c0e2d97502e9","Type":"ContainerStarted","Data":"0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27"}
Aug 04 19:18:36.498057 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:36.498062    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f" event={"ID":"9eae0c97-a0ca-4918-9f72-c0e2d97502e9","Type":"ContainerStarted","Data":"82c4371c807e468428ea61e9218c018277fdb1fa4bb3faa2396d4822d45ce638"}
Aug 04 19:18:36.518440 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:36.518386    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f" podStartSLOduration=1.518365845 podStartE2EDuration="1.518365845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:18:36.517396591 +0000 UTC m=+3058.384311691" watchObservedRunningTime="2026-08-04 19:18:36.518365845 +0000 UTC m=+3058.385280946"
Aug 04 19:18:45.792835 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:45.792794    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:45.793295 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:45.792848    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:45.804896 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:45.804871    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:18:46.556256 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:18:46.556229    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:19:43.519990 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:43.519957    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:19:44.393849 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:44.393802    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"]
Aug 04 19:19:44.416547 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:44.416520    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:19:59.299062 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.299032    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:19:59.411947 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.411880    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f" podUID="9eae0c97-a0ca-4918-9f72-c0e2d97502e9" containerName="main" containerID="cri-o://0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27" gracePeriod=45
Aug 04 19:19:59.651872 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.651849    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:19:59.775181 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775151    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tbwz\" (UniqueName: \"kubernetes.io/projected/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-kube-api-access-8tbwz\") pod \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") "
Aug 04 19:19:59.775341 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775241    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tls-certs\") pod \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") "
Aug 04 19:19:59.775341 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775268    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tmp-dir\") pod \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") "
Aug 04 19:19:59.775341 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775297    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-model-cache\") pod \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") "
Aug 04 19:19:59.775518 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775433    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-home\") pod \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") "
Aug 04 19:19:59.775518 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775469    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-dshm\") pod \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\" (UID: \"9eae0c97-a0ca-4918-9f72-c0e2d97502e9\") "
Aug 04 19:19:59.775518 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775485    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9eae0c97-a0ca-4918-9f72-c0e2d97502e9" (UID: "9eae0c97-a0ca-4918-9f72-c0e2d97502e9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:19:59.775704 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775667    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-model-cache" (OuterVolumeSpecName: "model-cache") pod "9eae0c97-a0ca-4918-9f72-c0e2d97502e9" (UID: "9eae0c97-a0ca-4918-9f72-c0e2d97502e9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:19:59.775753 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775707    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-home" (OuterVolumeSpecName: "home") pod "9eae0c97-a0ca-4918-9f72-c0e2d97502e9" (UID: "9eae0c97-a0ca-4918-9f72-c0e2d97502e9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:19:59.775801 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775785    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-home\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:19:59.775842 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775804    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tmp-dir\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:19:59.775842 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.775819    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-model-cache\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:19:59.777442 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.777419    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-dshm" (OuterVolumeSpecName: "dshm") pod "9eae0c97-a0ca-4918-9f72-c0e2d97502e9" (UID: "9eae0c97-a0ca-4918-9f72-c0e2d97502e9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:19:59.777543 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.777464    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9eae0c97-a0ca-4918-9f72-c0e2d97502e9" (UID: "9eae0c97-a0ca-4918-9f72-c0e2d97502e9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 19:19:59.777543 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.777477    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-kube-api-access-8tbwz" (OuterVolumeSpecName: "kube-api-access-8tbwz") pod "9eae0c97-a0ca-4918-9f72-c0e2d97502e9" (UID: "9eae0c97-a0ca-4918-9f72-c0e2d97502e9"). InnerVolumeSpecName "kube-api-access-8tbwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:19:59.877003 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.876931    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-tls-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:19:59.877003 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.876955    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-dshm\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:19:59.877003 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.876964    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tbwz\" (UniqueName: \"kubernetes.io/projected/9eae0c97-a0ca-4918-9f72-c0e2d97502e9-kube-api-access-8tbwz\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:19:59.895827 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.895798    2578 generic.go:356] "Generic (PLEG): container finished" podID="9eae0c97-a0ca-4918-9f72-c0e2d97502e9" containerID="0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27" exitCode=0
Aug 04 19:19:59.895958 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.895846    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"
Aug 04 19:19:59.895958 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.895884    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f" event={"ID":"9eae0c97-a0ca-4918-9f72-c0e2d97502e9","Type":"ContainerDied","Data":"0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27"}
Aug 04 19:19:59.895958 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.895921    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f" event={"ID":"9eae0c97-a0ca-4918-9f72-c0e2d97502e9","Type":"ContainerDied","Data":"82c4371c807e468428ea61e9218c018277fdb1fa4bb3faa2396d4822d45ce638"}
Aug 04 19:19:59.895958 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.895938    2578 scope.go:117] "RemoveContainer" containerID="0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27"
Aug 04 19:19:59.904479 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.904461    2578 scope.go:117] "RemoveContainer" containerID="0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27"
Aug 04 19:19:59.904741 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:19:59.904721    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27\": container with ID starting with 0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27 not found: ID does not exist" containerID="0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27"
Aug 04 19:19:59.904813 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.904752    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27"} err="failed to get container status \"0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27\": rpc error: code = NotFound desc = could not find container \"0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27\": container with ID starting with 0a6aaa406aabee9f209f27ebc60fa3d381d5c15982bb1984b4f2fb9d7c200f27 not found: ID does not exist"
Aug 04 19:19:59.917418 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.917397    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"]
Aug 04 19:19:59.922073 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:19:59.922050    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/sub-test-model-1a00f6-kserve-bc8b88f5f-2ln6f"]
Aug 04 19:20:00.802793 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:00.802696    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9eae0c97-a0ca-4918-9f72-c0e2d97502e9" path="/var/lib/kubelet/pods/9eae0c97-a0ca-4918-9f72-c0e2d97502e9/volumes"
Aug 04 19:20:44.879287 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.879249    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"]
Aug 04 19:20:44.879855 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.879801    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9eae0c97-a0ca-4918-9f72-c0e2d97502e9" containerName="main"
Aug 04 19:20:44.879855 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.879822    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9eae0c97-a0ca-4918-9f72-c0e2d97502e9" containerName="main"
Aug 04 19:20:44.879978 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.879953    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9eae0c97-a0ca-4918-9f72-c0e2d97502e9" containerName="main"
Aug 04 19:20:44.884273 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.884249    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:44.888190 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.888168    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1a00f6\"/\"openshift-service-ca.crt\""
Aug 04 19:20:44.888441 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.888164    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1a00f6\"/\"rate-test-model-1a00f6-kserve-self-signed-certs\""
Aug 04 19:20:44.888932 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.888200    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1a00f6\"/\"kube-root-ca.crt\""
Aug 04 19:20:44.888932 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.888632    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1a00f6\"/\"default-dockercfg-7snqf\""
Aug 04 19:20:44.891252 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.891227    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"]
Aug 04 19:20:44.923483 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.923457    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:20:44.974300 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.974270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-dshm\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:44.974423 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.974335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-model-cache\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:44.974423 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.974390    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b610613e-58e8-4735-b5b2-470a61d6a68f-tls-certs\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:44.974515 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.974442    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-tmp-dir\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:44.974515 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.974493    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dzmt\" (UniqueName: \"kubernetes.io/projected/b610613e-58e8-4735-b5b2-470a61d6a68f-kube-api-access-8dzmt\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:44.974634 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:44.974519    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-home\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.075570 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.075540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-dshm\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.075743 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.075691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-model-cache\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.075743 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.075719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b610613e-58e8-4735-b5b2-470a61d6a68f-tls-certs\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.075743 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.075737    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-tmp-dir\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.075899 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.075768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dzmt\" (UniqueName: \"kubernetes.io/projected/b610613e-58e8-4735-b5b2-470a61d6a68f-kube-api-access-8dzmt\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.075899 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.075800    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-home\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.076108 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.076075    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-model-cache\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.076108 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.076087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-tmp-dir\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.076306 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.076155    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-home\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.077759 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.077739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-dshm\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.078026 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.078008    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b610613e-58e8-4735-b5b2-470a61d6a68f-tls-certs\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.085254 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.085229    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dzmt\" (UniqueName: \"kubernetes.io/projected/b610613e-58e8-4735-b5b2-470a61d6a68f-kube-api-access-8dzmt\") pod \"rate-test-model-1a00f6-kserve-f8d666b75-ttvgz\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") " pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.195755 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.195675    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:45.333821 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:45.333796    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"]
Aug 04 19:20:45.335994 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:20:45.335961    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb610613e_58e8_4735_b5b2_470a61d6a68f.slice/crio-c2145f9d170bc43226abde27e34a1c91e46bcb144b21a635eb3b64b65a003e88 WatchSource:0}: Error finding container c2145f9d170bc43226abde27e34a1c91e46bcb144b21a635eb3b64b65a003e88: Status 404 returned error can't find the container with id c2145f9d170bc43226abde27e34a1c91e46bcb144b21a635eb3b64b65a003e88
Aug 04 19:20:46.123799 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:46.123761    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz" event={"ID":"b610613e-58e8-4735-b5b2-470a61d6a68f","Type":"ContainerStarted","Data":"3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2"}
Aug 04 19:20:46.124165 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:46.123806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz" event={"ID":"b610613e-58e8-4735-b5b2-470a61d6a68f","Type":"ContainerStarted","Data":"c2145f9d170bc43226abde27e34a1c91e46bcb144b21a635eb3b64b65a003e88"}
Aug 04 19:20:46.145045 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:46.144995    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz" podStartSLOduration=2.144980558 podStartE2EDuration="2.144980558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:20:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:20:46.143265975 +0000 UTC m=+3188.010181077" watchObservedRunningTime="2026-08-04 19:20:46.144980558 +0000 UTC m=+3188.011895682"
Aug 04 19:20:55.196174 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:55.196141    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:55.196607 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:55.196186    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:55.208338 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:55.208313    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:20:56.180433 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:20:56.180404    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:21:02.349222 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:21:02.349184    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:21:04.994911 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:21:04.994877    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:21:51.996418 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:21:51.996375    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:21:52.883886 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:21:52.883848    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"]
Aug 04 19:21:52.904500 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:21:52.904472    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:07.903006 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:07.902957    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz" podUID="b610613e-58e8-4735-b5b2-470a61d6a68f" containerName="main" containerID="cri-o://3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2" gracePeriod=45
Aug 04 19:22:08.153663 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.153593    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:22:08.222264 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222235    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-model-cache\") pod \"b610613e-58e8-4735-b5b2-470a61d6a68f\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") "
Aug 04 19:22:08.222447 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222273    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-dshm\") pod \"b610613e-58e8-4735-b5b2-470a61d6a68f\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") "
Aug 04 19:22:08.222447 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222323    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-home\") pod \"b610613e-58e8-4735-b5b2-470a61d6a68f\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") "
Aug 04 19:22:08.222447 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222349    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b610613e-58e8-4735-b5b2-470a61d6a68f-tls-certs\") pod \"b610613e-58e8-4735-b5b2-470a61d6a68f\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") "
Aug 04 19:22:08.222447 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222376    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-tmp-dir\") pod \"b610613e-58e8-4735-b5b2-470a61d6a68f\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") "
Aug 04 19:22:08.222447 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222426    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dzmt\" (UniqueName: \"kubernetes.io/projected/b610613e-58e8-4735-b5b2-470a61d6a68f-kube-api-access-8dzmt\") pod \"b610613e-58e8-4735-b5b2-470a61d6a68f\" (UID: \"b610613e-58e8-4735-b5b2-470a61d6a68f\") "
Aug 04 19:22:08.222753 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222602    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-model-cache" (OuterVolumeSpecName: "model-cache") pod "b610613e-58e8-4735-b5b2-470a61d6a68f" (UID: "b610613e-58e8-4735-b5b2-470a61d6a68f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:22:08.222753 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222645    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-home" (OuterVolumeSpecName: "home") pod "b610613e-58e8-4735-b5b2-470a61d6a68f" (UID: "b610613e-58e8-4735-b5b2-470a61d6a68f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:22:08.222753 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222720    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b610613e-58e8-4735-b5b2-470a61d6a68f" (UID: "b610613e-58e8-4735-b5b2-470a61d6a68f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:22:08.222898 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222810    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-model-cache\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:22:08.222898 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222829    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-home\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:22:08.222898 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.222843    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-tmp-dir\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:22:08.224585 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.224549    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b610613e-58e8-4735-b5b2-470a61d6a68f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b610613e-58e8-4735-b5b2-470a61d6a68f" (UID: "b610613e-58e8-4735-b5b2-470a61d6a68f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 19:22:08.224663 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.224628    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-dshm" (OuterVolumeSpecName: "dshm") pod "b610613e-58e8-4735-b5b2-470a61d6a68f" (UID: "b610613e-58e8-4735-b5b2-470a61d6a68f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:22:08.224663 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.224644    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b610613e-58e8-4735-b5b2-470a61d6a68f-kube-api-access-8dzmt" (OuterVolumeSpecName: "kube-api-access-8dzmt") pod "b610613e-58e8-4735-b5b2-470a61d6a68f" (UID: "b610613e-58e8-4735-b5b2-470a61d6a68f"). InnerVolumeSpecName "kube-api-access-8dzmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:22:08.323720 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.323691    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b610613e-58e8-4735-b5b2-470a61d6a68f-dshm\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:22:08.323720 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.323716    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b610613e-58e8-4735-b5b2-470a61d6a68f-tls-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:22:08.323720 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.323728    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dzmt\" (UniqueName: \"kubernetes.io/projected/b610613e-58e8-4735-b5b2-470a61d6a68f-kube-api-access-8dzmt\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:22:08.538819 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.538781    2578 generic.go:356] "Generic (PLEG): container finished" podID="b610613e-58e8-4735-b5b2-470a61d6a68f" containerID="3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2" exitCode=0
Aug 04 19:22:08.538998 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.538823    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"
Aug 04 19:22:08.538998 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.538841    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz" event={"ID":"b610613e-58e8-4735-b5b2-470a61d6a68f","Type":"ContainerDied","Data":"3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2"}
Aug 04 19:22:08.538998 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.538880    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz" event={"ID":"b610613e-58e8-4735-b5b2-470a61d6a68f","Type":"ContainerDied","Data":"c2145f9d170bc43226abde27e34a1c91e46bcb144b21a635eb3b64b65a003e88"}
Aug 04 19:22:08.538998 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.538897    2578 scope.go:117] "RemoveContainer" containerID="3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2"
Aug 04 19:22:08.548096 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.548078    2578 scope.go:117] "RemoveContainer" containerID="3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2"
Aug 04 19:22:08.548327 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:22:08.548310    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2\": container with ID starting with 3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2 not found: ID does not exist" containerID="3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2"
Aug 04 19:22:08.548384 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.548333    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2"} err="failed to get container status \"3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2\": rpc error: code = NotFound desc = could not find container \"3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2\": container with ID starting with 3baaf1b193e9bbee3bd29aecd951ee3de68c96618802ed8a615ef00810f3b9b2 not found: ID does not exist"
Aug 04 19:22:08.561915 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.561879    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"]
Aug 04 19:22:08.568115 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.568094    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1a00f6/rate-test-model-1a00f6-kserve-f8d666b75-ttvgz"]
Aug 04 19:22:08.803324 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:08.803244    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b610613e-58e8-4735-b5b2-470a61d6a68f" path="/var/lib/kubelet/pods/b610613e-58e8-4735-b5b2-470a61d6a68f/volumes"
Aug 04 19:22:12.013114 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:12.013082    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:12.339324 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:12.339290    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:12.473879 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:12.473845    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:15.295755 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:15.295718    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:15.398639 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:15.398603    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:16.452468 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:16.452431    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:16.813570 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:16.813538    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:17.042911 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.042877    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:17.140004 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.139907    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:17.143728 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.143702    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"]
Aug 04 19:22:17.144071 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.144060    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b610613e-58e8-4735-b5b2-470a61d6a68f" containerName="main"
Aug 04 19:22:17.144120 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.144073    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b610613e-58e8-4735-b5b2-470a61d6a68f" containerName="main"
Aug 04 19:22:17.144160 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.144125    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b610613e-58e8-4735-b5b2-470a61d6a68f" containerName="main"
Aug 04 19:22:17.148482 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.148464    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:22:17.155848 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.155831    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f3a5e8\""
Aug 04 19:22:17.171343 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.171317    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"]
Aug 04 19:22:17.306646 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.306613    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h\" (UID: \"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:22:17.306800 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.306736    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2trm\" (UniqueName: \"kubernetes.io/projected/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-kube-api-access-x2trm\") pod \"payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h\" (UID: \"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:22:17.407756 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.407673    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2trm\" (UniqueName: \"kubernetes.io/projected/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-kube-api-access-x2trm\") pod \"payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h\" (UID: \"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:22:17.407756 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.407734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h\" (UID: \"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:22:17.408400 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.408381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h\" (UID: \"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:22:17.415770 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.415745    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2trm\" (UniqueName: \"kubernetes.io/projected/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-kube-api-access-x2trm\") pod \"payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h\" (UID: \"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:22:17.457835 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.457804    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:22:17.594432 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:17.594405    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"]
Aug 04 19:22:17.596432 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:22:17.596404    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcbe73c7_ebc2_48e2_95c9_81b49405cdc6.slice/crio-49c6d56e33bfc056393859f12fdb92087caabe760e3fabc348ff61577df0b6b3 WatchSource:0}: Error finding container 49c6d56e33bfc056393859f12fdb92087caabe760e3fabc348ff61577df0b6b3: Status 404 returned error can't find the container with id 49c6d56e33bfc056393859f12fdb92087caabe760e3fabc348ff61577df0b6b3
Aug 04 19:22:18.589911 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:18.589874    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h" event={"ID":"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6","Type":"ContainerStarted","Data":"c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590"}
Aug 04 19:22:18.590373 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:18.589931    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h" event={"ID":"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6","Type":"ContainerStarted","Data":"49c6d56e33bfc056393859f12fdb92087caabe760e3fabc348ff61577df0b6b3"}
Aug 04 19:22:18.590373 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:18.590042    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:22:18.608321 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:18.608276    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h" podStartSLOduration=1.608262141 podStartE2EDuration="1.608262141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:22:18.606701987 +0000 UTC m=+3280.473617089" watchObservedRunningTime="2026-08-04 19:22:18.608262141 +0000 UTC m=+3280.475177242"
Aug 04 19:22:29.598049 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:29.598007    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:22:40.288216 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:40.288182    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:40.939492 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:40.939459    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:41.211066 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.210994    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"]
Aug 04 19:22:41.215918 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.215894    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.219028 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.219005    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-f18332-openshift-default-dockercfg-mm8jv\""
Aug 04 19:22:41.228895 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.228873    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"]
Aug 04 19:22:41.241495 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.241466    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:41.310304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.307392    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-token\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.310304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.307521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-data\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.310304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.307609    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istiod-ca-cert\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.310304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.307723    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-credential-socket\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.310304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.307756    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-certs\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.310304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.307790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-envoy\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.310304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.307821    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtrjh\" (UniqueName: \"kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-kube-api-access-qtrjh\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.310304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.307948    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-socket\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.310304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.307981    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-podinfo\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.409308 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.409256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-credential-socket\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.409308 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.409312    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-certs\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.409571 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.409340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-envoy\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.409571 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.409363    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtrjh\" (UniqueName: \"kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-kube-api-access-qtrjh\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.409571 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.409435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-socket\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.409571 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.409456    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-podinfo\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.409571 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.409496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-token\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.409571 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.409550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-data\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.409825 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.409626    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istiod-ca-cert\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.409825 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.409795    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-credential-socket\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.410049 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.410026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-certs\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.410124 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.410073    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-socket\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.410173 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.410128    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-data\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.410646 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.410629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istiod-ca-cert\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.411736 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.411717    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-envoy\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.411892 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.411873    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-podinfo\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.420260 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.420233    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-token\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.420424 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.420407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtrjh\" (UniqueName: \"kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-kube-api-access-qtrjh\") pod \"e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") " pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.526203 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.526116    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:41.667037 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.667010    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"]
Aug 04 19:22:41.668754 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:22:41.668725    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d40c5ac_89c4_46b4_bb40_c7b44f1efcec.slice/crio-c5b5b2f7e4411085240628b17839ba9f5a13d5c0d97e1f7edb6ec16af54f6508 WatchSource:0}: Error finding container c5b5b2f7e4411085240628b17839ba9f5a13d5c0d97e1f7edb6ec16af54f6508: Status 404 returned error can't find the container with id c5b5b2f7e4411085240628b17839ba9f5a13d5c0d97e1f7edb6ec16af54f6508
Aug 04 19:22:41.671117 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.671073    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:22:41.671204 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.671170    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:22:41.671245 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.671216    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:22:41.700991 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.700960    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" event={"ID":"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec","Type":"ContainerStarted","Data":"c5b5b2f7e4411085240628b17839ba9f5a13d5c0d97e1f7edb6ec16af54f6508"}
Aug 04 19:22:41.725735 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:41.725707    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:42.707799 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:42.707768    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" event={"ID":"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec","Type":"ContainerStarted","Data":"be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22"}
Aug 04 19:22:42.729675 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:42.729626    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" podStartSLOduration=1.729607012 podStartE2EDuration="1.729607012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:22:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:22:42.728232685 +0000 UTC m=+3304.595147787" watchObservedRunningTime="2026-08-04 19:22:42.729607012 +0000 UTC m=+3304.596522113"
Aug 04 19:22:43.527087 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:43.527048    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:43.532377 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:43.532347    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:43.713199 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:43.713169    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:43.714105 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:43.714087    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:22:44.646380 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:44.646346    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:44.736495 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:44.736466    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:45.748722 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:45.748684    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:46.062120 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.062086    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:46.293767 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.293735    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:46.389276 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.389195    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:22:46.415296 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.415265    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"]
Aug 04 19:22:46.418752 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.418735    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:22:46.421642 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.421624    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f18332-dockercfg-5gz27\""
Aug 04 19:22:46.421765 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.421700    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f18332\""
Aug 04 19:22:46.433362 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.433337    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"]
Aug 04 19:22:46.456051 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.456024    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bhs8\" (UniqueName: \"kubernetes.io/projected/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-kube-api-access-2bhs8\") pod \"payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6\" (UID: \"8cfe8865-0d4a-46a4-a38a-b565e1f83f33\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:22:46.456202 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.456074    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6\" (UID: \"8cfe8865-0d4a-46a4-a38a-b565e1f83f33\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:22:46.557631 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.557595    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bhs8\" (UniqueName: \"kubernetes.io/projected/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-kube-api-access-2bhs8\") pod \"payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6\" (UID: \"8cfe8865-0d4a-46a4-a38a-b565e1f83f33\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:22:46.557802 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.557647    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6\" (UID: \"8cfe8865-0d4a-46a4-a38a-b565e1f83f33\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:22:46.558296 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.558276    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6\" (UID: \"8cfe8865-0d4a-46a4-a38a-b565e1f83f33\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:22:46.567102 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.567071    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bhs8\" (UniqueName: \"kubernetes.io/projected/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-kube-api-access-2bhs8\") pod \"payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6\" (UID: \"8cfe8865-0d4a-46a4-a38a-b565e1f83f33\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:22:46.730811 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.730728    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:22:46.869210 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:46.869186    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"]
Aug 04 19:22:46.871393 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:22:46.871366    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cfe8865_0d4a_46a4_a38a_b565e1f83f33.slice/crio-3a4449a8c92d1ad7a8e3d9626e1e6dae261728daaf073a0a3e8379bff3dd9d95 WatchSource:0}: Error finding container 3a4449a8c92d1ad7a8e3d9626e1e6dae261728daaf073a0a3e8379bff3dd9d95: Status 404 returned error can't find the container with id 3a4449a8c92d1ad7a8e3d9626e1e6dae261728daaf073a0a3e8379bff3dd9d95
Aug 04 19:22:47.736474 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:47.736428    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6" event={"ID":"8cfe8865-0d4a-46a4-a38a-b565e1f83f33","Type":"ContainerStarted","Data":"fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c"}
Aug 04 19:22:47.736693 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:47.736529    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6" event={"ID":"8cfe8865-0d4a-46a4-a38a-b565e1f83f33","Type":"ContainerStarted","Data":"3a4449a8c92d1ad7a8e3d9626e1e6dae261728daaf073a0a3e8379bff3dd9d95"}
Aug 04 19:22:47.736693 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:47.736564    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:22:47.753224 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:47.753174    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6" podStartSLOduration=1.753131889 podStartE2EDuration="1.753131889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:22:47.752040897 +0000 UTC m=+3309.618955998" watchObservedRunningTime="2026-08-04 19:22:47.753131889 +0000 UTC m=+3309.620046994"
Aug 04 19:22:58.714388 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:58.714355    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Aug 04 19:22:58.714888 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:58.714413    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" podUID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Aug 04 19:22:58.742962 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:58.742934    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:22:58.793211 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:58.793179    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerID="be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22" exitCode=0
Aug 04 19:22:58.793363 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:58.793263    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" event={"ID":"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec","Type":"ContainerDied","Data":"be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22"}
Aug 04 19:22:58.793698 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:58.793683    2578 scope.go:117] "RemoveContainer" containerID="be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22"
Aug 04 19:22:58.794895 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:58.794862    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:22:58.794986 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:58.794932    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:22:58.794986 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:58.794967    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:22:59.799818 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:22:59.799785    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" event={"ID":"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec","Type":"ContainerStarted","Data":"188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d"}
Aug 04 19:23:00.526710 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:00.526670    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:23:00.526921 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:00.526832    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Aug 04 19:23:00.526921 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:00.526888    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" podUID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Aug 04 19:23:01.526795 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:01.526752    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Aug 04 19:23:01.527347 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:01.526827    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" podUID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Aug 04 19:23:02.526506 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:02.526462    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Aug 04 19:23:02.526725 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:02.526535    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" podUID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Aug 04 19:23:03.526733 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:03.526691    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Aug 04 19:23:03.527188 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:03.526754    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" podUID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Aug 04 19:23:04.531480 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:04.531439    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:23:04.531954 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:04.531853    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:23:04.532683 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:04.532660    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:23:14.704594 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:14.704543    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:23:21.585658 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:21.585626    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6_8cfe8865-0d4a-46a4-a38a-b565e1f83f33/payload-processing/0.log"
Aug 04 19:23:28.999818 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:28.999788    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:23:48.454556 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:48.454518    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:23:48.552268 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:23:48.552237    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:06.491920 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:06.491890    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6_8cfe8865-0d4a-46a4-a38a-b565e1f83f33/payload-processing/0.log"
Aug 04 19:24:06.763072 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:06.762990    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:07.040407 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.040377    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"]
Aug 04 19:24:07.043909 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.043881    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.046647 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.046619    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-249eef-openshift-default-dockercfg-4ccps\""
Aug 04 19:24:07.056467 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.056448    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"]
Aug 04 19:24:07.067960 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.067937    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:07.170531 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.170500    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-socket\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.170715 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.170543    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-data\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.170715 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.170565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-certs\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.170715 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.170645    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m27t7\" (UniqueName: \"kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-kube-api-access-m27t7\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.170715 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.170679    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-token\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.170715 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.170696    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.170886 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.170782    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-podinfo\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.170886 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.170809    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-credential-socket\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.170886 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.170824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-envoy\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.271707 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.271682    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-socket\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.271874 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.271719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-data\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.271874 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.271741    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-certs\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.271874 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.271766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m27t7\" (UniqueName: \"kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-kube-api-access-m27t7\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.271874 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.271785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-token\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.271874 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.271800    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.271874 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.271834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-podinfo\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.271874 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.271855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-credential-socket\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.272232 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.271877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-envoy\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.272232 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.272112    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-socket\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.272232 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.272149    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-data\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.272396 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.272231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-certs\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.272453 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.272416    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-credential-socket\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.272542 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.272520    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.274043 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.274024    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-envoy\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.274223 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.274207    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-podinfo\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.280045 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.280016    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-token\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.280168 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.280150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m27t7\" (UniqueName: \"kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-kube-api-access-m27t7\") pod \"e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") " pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.328443 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.328328    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:07.354061 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.354034    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:07.492287 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.492258    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"]
Aug 04 19:24:07.493854 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:24:07.493822    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c4764ff_12b3_4522_a890_039ce1fe1f8d.slice/crio-912779e2fa550a49d35d4e4966c79dfe3b7d1b7552283e3dad27af95d74c811a WatchSource:0}: Error finding container 912779e2fa550a49d35d4e4966c79dfe3b7d1b7552283e3dad27af95d74c811a: Status 404 returned error can't find the container with id 912779e2fa550a49d35d4e4966c79dfe3b7d1b7552283e3dad27af95d74c811a
Aug 04 19:24:07.495972 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.495939    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:24:07.496050 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.496001    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:24:07.496050 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:07.496031    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:24:08.146919 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:08.146884    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" event={"ID":"7c4764ff-12b3-4522-a890-039ce1fe1f8d","Type":"ContainerStarted","Data":"0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d"}
Aug 04 19:24:08.147084 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:08.146937    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" event={"ID":"7c4764ff-12b3-4522-a890-039ce1fe1f8d","Type":"ContainerStarted","Data":"912779e2fa550a49d35d4e4966c79dfe3b7d1b7552283e3dad27af95d74c811a"}
Aug 04 19:24:08.168538 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:08.168489    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" podStartSLOduration=1.168473881 podStartE2EDuration="1.168473881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 19:24:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:24:08.166391664 +0000 UTC m=+3390.033306764" watchObservedRunningTime="2026-08-04 19:24:08.168473881 +0000 UTC m=+3390.035388981"
Aug 04 19:24:08.354884 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:08.354854    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:08.359834 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:08.359806    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:09.151422 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:09.151396    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:09.152209 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:09.152191    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:10.112657 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:10.112625    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:10.200489 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:10.200459    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:11.214983 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:11.214944    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:11.393322 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:11.393289    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:11.712923 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:11.712888    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:11.817270 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:11.817244    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:25.193749 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:25.193703    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Aug 04 19:24:25.194144 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:25.193783    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Aug 04 19:24:26.240783 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:26.240751    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerID="0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d" exitCode=0
Aug 04 19:24:26.241175 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:26.240823    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" event={"ID":"7c4764ff-12b3-4522-a890-039ce1fe1f8d","Type":"ContainerDied","Data":"0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d"}
Aug 04 19:24:26.241259 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:26.241243    2578 scope.go:117] "RemoveContainer" containerID="0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d"
Aug 04 19:24:26.242319 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:26.242289    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:24:26.242431 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:26.242372    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:24:26.242431 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:26.242419    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 19:24:27.247068 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:27.247037    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" event={"ID":"7c4764ff-12b3-4522-a890-039ce1fe1f8d","Type":"ContainerStarted","Data":"9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32"}
Aug 04 19:24:27.354506 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:27.354471    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:27.354771 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:27.354746    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Aug 04 19:24:27.354830 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:27.354810    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Aug 04 19:24:28.354989 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:28.354957    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Aug 04 19:24:28.355344 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:28.355006    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Aug 04 19:24:29.354508 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:29.354473    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Aug 04 19:24:29.354706 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:29.354524    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Aug 04 19:24:30.354318 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:30.354285    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Aug 04 19:24:30.354780 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:30.354343    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Aug 04 19:24:31.355459 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:31.355423    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Aug 04 19:24:31.355912 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:31.355480    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Aug 04 19:24:32.359726 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:32.359685    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:32.360201 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:32.359985    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:32.360798 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:32.360775    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:35.106522 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:35.106490    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:38.688625 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:38.688592    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:41.811525 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:41.811490    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:41.913378 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:41.913344    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:42.013553 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:42.013517    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:42.097298 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:42.097208    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:42.501720 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:42.501632    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:48.495689 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:48.495654    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:48.579814 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:48.579782    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:48.697153 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:48.697125    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"]
Aug 04 19:24:48.697341 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:48.697316    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy" containerID="cri-o://9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32" gracePeriod=30
Aug 04 19:24:48.728943 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:48.728908    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:48.933179 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:48.933128    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:50.245475 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:50.245428    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:24:53.942360 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:53.942337    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:54.080146 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080107    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m27t7\" (UniqueName: \"kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-kube-api-access-m27t7\") pod \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") "
Aug 04 19:24:54.080331 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080156    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-envoy\") pod \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") "
Aug 04 19:24:54.080331 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080185    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-data\") pod \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") "
Aug 04 19:24:54.080331 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080209    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-certs\") pod \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") "
Aug 04 19:24:54.080331 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080230    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-token\") pod \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") "
Aug 04 19:24:54.080331 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080253    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-credential-socket\") pod \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") "
Aug 04 19:24:54.080613 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080412    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-podinfo\") pod \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") "
Aug 04 19:24:54.080613 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080496    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-socket\") pod \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") "
Aug 04 19:24:54.080613 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080520    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istiod-ca-cert\") pod \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\" (UID: \"7c4764ff-12b3-4522-a890-039ce1fe1f8d\") "
Aug 04 19:24:54.080613 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080529    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-data" (OuterVolumeSpecName: "istio-data") pod "7c4764ff-12b3-4522-a890-039ce1fe1f8d" (UID: "7c4764ff-12b3-4522-a890-039ce1fe1f8d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:24:54.080613 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080558    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7c4764ff-12b3-4522-a890-039ce1fe1f8d" (UID: "7c4764ff-12b3-4522-a890-039ce1fe1f8d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:24:54.080858 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080799    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7c4764ff-12b3-4522-a890-039ce1fe1f8d" (UID: "7c4764ff-12b3-4522-a890-039ce1fe1f8d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:24:54.080858 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080844    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7c4764ff-12b3-4522-a890-039ce1fe1f8d" (UID: "7c4764ff-12b3-4522-a890-039ce1fe1f8d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:24:54.080986 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080970    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:24:54.081040 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080986    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-workload-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:24:54.081040 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080996    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istiod-ca-cert\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:24:54.081040 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.080995    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7c4764ff-12b3-4522-a890-039ce1fe1f8d" (UID: "7c4764ff-12b3-4522-a890-039ce1fe1f8d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:24:54.081040 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.081008    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-data\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:24:54.082702 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.082683    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-kube-api-access-m27t7" (OuterVolumeSpecName: "kube-api-access-m27t7") pod "7c4764ff-12b3-4522-a890-039ce1fe1f8d" (UID: "7c4764ff-12b3-4522-a890-039ce1fe1f8d"). InnerVolumeSpecName "kube-api-access-m27t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:24:54.082807 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.082768    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7c4764ff-12b3-4522-a890-039ce1fe1f8d" (UID: "7c4764ff-12b3-4522-a890-039ce1fe1f8d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:24:54.082870 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.082848    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-token" (OuterVolumeSpecName: "istio-token") pod "7c4764ff-12b3-4522-a890-039ce1fe1f8d" (UID: "7c4764ff-12b3-4522-a890-039ce1fe1f8d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:24:54.082870 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.082859    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7c4764ff-12b3-4522-a890-039ce1fe1f8d" (UID: "7c4764ff-12b3-4522-a890-039ce1fe1f8d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 19:24:54.181511 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.181485    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-podinfo\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:24:54.181511 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.181508    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m27t7\" (UniqueName: \"kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-kube-api-access-m27t7\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:24:54.181672 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.181518    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-envoy\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:24:54.181672 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.181526    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c4764ff-12b3-4522-a890-039ce1fe1f8d-istio-token\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:24:54.181672 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.181534    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c4764ff-12b3-4522-a890-039ce1fe1f8d-credential-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:24:54.380938 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.380865    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerID="9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32" exitCode=0
Aug 04 19:24:54.380938 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.380919    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" event={"ID":"7c4764ff-12b3-4522-a890-039ce1fe1f8d","Type":"ContainerDied","Data":"9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32"}
Aug 04 19:24:54.380938 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.380927    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"
Aug 04 19:24:54.381167 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.380970    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd" event={"ID":"7c4764ff-12b3-4522-a890-039ce1fe1f8d","Type":"ContainerDied","Data":"912779e2fa550a49d35d4e4966c79dfe3b7d1b7552283e3dad27af95d74c811a"}
Aug 04 19:24:54.381167 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.380999    2578 scope.go:117] "RemoveContainer" containerID="9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32"
Aug 04 19:24:54.389779 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.389762    2578 scope.go:117] "RemoveContainer" containerID="0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d"
Aug 04 19:24:54.398070 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.398053    2578 scope.go:117] "RemoveContainer" containerID="9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32"
Aug 04 19:24:54.398310 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:24:54.398287    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32\": container with ID starting with 9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32 not found: ID does not exist" containerID="9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32"
Aug 04 19:24:54.398381 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.398322    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32"} err="failed to get container status \"9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32\": rpc error: code = NotFound desc = could not find container \"9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32\": container with ID starting with 9be34083fa4ed5d59049d88137471b54589e9537e3715e3259b3303f06857a32 not found: ID does not exist"
Aug 04 19:24:54.398381 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.398342    2578 scope.go:117] "RemoveContainer" containerID="0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d"
Aug 04 19:24:54.398549 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:24:54.398534    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d\": container with ID starting with 0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d not found: ID does not exist" containerID="0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d"
Aug 04 19:24:54.398630 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.398551    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d"} err="failed to get container status \"0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d\": rpc error: code = NotFound desc = could not find container \"0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d\": container with ID starting with 0402b6c6997d1ccc7af60035d121d8d8cddc2d7cdcf54c9d558ed42a4f86027d not found: ID does not exist"
Aug 04 19:24:54.405467 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.405445    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"]
Aug 04 19:24:54.410572 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.410549    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-249eef-openshift-default-5f45875494-cshbd"]
Aug 04 19:24:54.802403 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:24:54.802363    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" path="/var/lib/kubelet/pods/7c4764ff-12b3-4522-a890-039ce1fe1f8d/volumes"
Aug 04 19:25:07.991927 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:07.991888    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:25:08.022504 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.021671    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"]
Aug 04 19:25:08.022504 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.021927    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h" podUID="dcbe73c7-ebc2-48e2-95c9-81b49405cdc6" containerName="payload-pre-processing" containerID="cri-o://c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590" gracePeriod=30
Aug 04 19:25:08.068364 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.068312    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:25:08.179152 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.179117    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:25:08.292543 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.292519    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:25:08.311787 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.311757    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2trm\" (UniqueName: \"kubernetes.io/projected/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-kube-api-access-x2trm\") pod \"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6\" (UID: \"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6\") "
Aug 04 19:25:08.311993 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.311967    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-plugins-config-volume\") pod \"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6\" (UID: \"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6\") "
Aug 04 19:25:08.312490 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.312455    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dcbe73c7-ebc2-48e2-95c9-81b49405cdc6" (UID: "dcbe73c7-ebc2-48e2-95c9-81b49405cdc6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:25:08.316305 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.316266    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-kube-api-access-x2trm" (OuterVolumeSpecName: "kube-api-access-x2trm") pod "dcbe73c7-ebc2-48e2-95c9-81b49405cdc6" (UID: "dcbe73c7-ebc2-48e2-95c9-81b49405cdc6"). InnerVolumeSpecName "kube-api-access-x2trm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:25:08.413747 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.413710    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2trm\" (UniqueName: \"kubernetes.io/projected/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-kube-api-access-x2trm\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:25:08.413747 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.413741    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:25:08.451993 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.451962    2578 generic.go:356] "Generic (PLEG): container finished" podID="dcbe73c7-ebc2-48e2-95c9-81b49405cdc6" containerID="c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590" exitCode=0
Aug 04 19:25:08.452168 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.452012    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"
Aug 04 19:25:08.452168 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.452034    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h" event={"ID":"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6","Type":"ContainerDied","Data":"c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590"}
Aug 04 19:25:08.452168 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.452100    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h" event={"ID":"dcbe73c7-ebc2-48e2-95c9-81b49405cdc6","Type":"ContainerDied","Data":"49c6d56e33bfc056393859f12fdb92087caabe760e3fabc348ff61577df0b6b3"}
Aug 04 19:25:08.452168 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.452127    2578 scope.go:117] "RemoveContainer" containerID="c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590"
Aug 04 19:25:08.461263 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.461246    2578 scope.go:117] "RemoveContainer" containerID="c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590"
Aug 04 19:25:08.461514 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:25:08.461496    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590\": container with ID starting with c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590 not found: ID does not exist" containerID="c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590"
Aug 04 19:25:08.461601 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.461527    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590"} err="failed to get container status \"c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590\": rpc error: code = NotFound desc = could not find container \"c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590\": container with ID starting with c5cb843fb0b9f5c50c0ce6e5b490df322cd1f73aeacb264a39a6fefb72421590 not found: ID does not exist"
Aug 04 19:25:08.474509 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.474480    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"]
Aug 04 19:25:08.479384 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.479365    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3a5e8-6dbf8b48cb-mmd2h"]
Aug 04 19:25:08.546846 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.546811    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:25:08.802558 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:08.802482    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcbe73c7-ebc2-48e2-95c9-81b49405cdc6" path="/var/lib/kubelet/pods/dcbe73c7-ebc2-48e2-95c9-81b49405cdc6/volumes"
Aug 04 19:25:13.581434 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:13.581399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:25:37.285372 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:37.285327    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:25:50.190753 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:50.190724    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:25:51.319001 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:51.318967    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:25:57.239261 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:25:57.239224    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:03.111639 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:03.111528    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:03.175649 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:03.175609    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:03.408933 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:03.408841    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:03.631029 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:03.630995    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:08.282885 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:08.282852    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:14.244114 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.244079    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:14.266324 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.266293    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"]
Aug 04 19:26:14.266533 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.266508    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6" podUID="8cfe8865-0d4a-46a4-a38a-b565e1f83f33" containerName="payload-processing" containerID="cri-o://fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c" gracePeriod=30
Aug 04 19:26:14.307827 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.307791    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:14.429924 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.429838    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:14.529447 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.529422    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:26:14.607786 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.607717    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:14.622690 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.622647    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-plugins-config-volume\") pod \"8cfe8865-0d4a-46a4-a38a-b565e1f83f33\" (UID: \"8cfe8865-0d4a-46a4-a38a-b565e1f83f33\") "
Aug 04 19:26:14.622853 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.622771    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bhs8\" (UniqueName: \"kubernetes.io/projected/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-kube-api-access-2bhs8\") pod \"8cfe8865-0d4a-46a4-a38a-b565e1f83f33\" (UID: \"8cfe8865-0d4a-46a4-a38a-b565e1f83f33\") "
Aug 04 19:26:14.623282 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.623255    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8cfe8865-0d4a-46a4-a38a-b565e1f83f33" (UID: "8cfe8865-0d4a-46a4-a38a-b565e1f83f33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:26:14.625001 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.624969    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-kube-api-access-2bhs8" (OuterVolumeSpecName: "kube-api-access-2bhs8") pod "8cfe8865-0d4a-46a4-a38a-b565e1f83f33" (UID: "8cfe8865-0d4a-46a4-a38a-b565e1f83f33"). InnerVolumeSpecName "kube-api-access-2bhs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:26:14.724541 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.724501    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:14.724541 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.724536    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bhs8\" (UniqueName: \"kubernetes.io/projected/8cfe8865-0d4a-46a4-a38a-b565e1f83f33-kube-api-access-2bhs8\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:14.771175 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.771091    2578 generic.go:356] "Generic (PLEG): container finished" podID="8cfe8865-0d4a-46a4-a38a-b565e1f83f33" containerID="fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c" exitCode=0
Aug 04 19:26:14.771175 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.771139    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6" event={"ID":"8cfe8865-0d4a-46a4-a38a-b565e1f83f33","Type":"ContainerDied","Data":"fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c"}
Aug 04 19:26:14.771365 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.771225    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6" event={"ID":"8cfe8865-0d4a-46a4-a38a-b565e1f83f33","Type":"ContainerDied","Data":"3a4449a8c92d1ad7a8e3d9626e1e6dae261728daaf073a0a3e8379bff3dd9d95"}
Aug 04 19:26:14.771365 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.771146    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"
Aug 04 19:26:14.771365 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.771248    2578 scope.go:117] "RemoveContainer" containerID="fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c"
Aug 04 19:26:14.780242 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.780232    2578 scope.go:117] "RemoveContainer" containerID="fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c"
Aug 04 19:26:14.780495 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:26:14.780474    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c\": container with ID starting with fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c not found: ID does not exist" containerID="fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c"
Aug 04 19:26:14.780570 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.780508    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c"} err="failed to get container status \"fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c\": rpc error: code = NotFound desc = could not find container \"fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c\": container with ID starting with fb26563f24930d48bdfaa77ad9d32bd77794eb9eb537769af614b33b40ddcf3c not found: ID does not exist"
Aug 04 19:26:14.792847 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.792825    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"]
Aug 04 19:26:14.802090 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:14.802067    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f18332-659cd5d9c8-v7hj6"]
Aug 04 19:26:15.041456 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:15.041426    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:16.803024 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:16.802989    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cfe8865-0d4a-46a4-a38a-b565e1f83f33" path="/var/lib/kubelet/pods/8cfe8865-0d4a-46a4-a38a-b565e1f83f33/volumes"
Aug 04 19:26:21.675465 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:21.675433    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:21.741812 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:21.741766    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:21.937103 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:21.937013    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"]
Aug 04 19:26:21.937298 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:21.937269    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" podUID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerName="istio-proxy" containerID="cri-o://188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d" gracePeriod=30
Aug 04 19:26:21.960091 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:21.960066    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:22.190067 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:22.189986    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:27.190361 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.190338    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:26:27.241600 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.241522    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtrjh\" (UniqueName: \"kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-kube-api-access-qtrjh\") pod \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") "
Aug 04 19:26:27.241600 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.241565    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-envoy\") pod \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") "
Aug 04 19:26:27.241803 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.241623    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-credential-socket\") pod \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") "
Aug 04 19:26:27.241803 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.241646    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-data\") pod \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") "
Aug 04 19:26:27.241803 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.241676    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-podinfo\") pod \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") "
Aug 04 19:26:27.241803 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.241702    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-token\") pod \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") "
Aug 04 19:26:27.241803 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.241725    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-socket\") pod \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") "
Aug 04 19:26:27.242046 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.241947    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" (UID: "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:26:27.242046 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.241965    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" (UID: "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:26:27.242046 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.241990    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" (UID: "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:26:27.242046 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.242030    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istiod-ca-cert\") pod \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") "
Aug 04 19:26:27.242246 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.242096    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-certs\") pod \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\" (UID: \"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec\") "
Aug 04 19:26:27.242402 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.242362    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" (UID: "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:26:27.242402 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.242371    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" (UID: "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:26:27.242525 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.242471    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:27.242525 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.242489    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istiod-ca-cert\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:27.242525 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.242504    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-workload-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:27.242525 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.242517    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-credential-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:27.242678 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.242530    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-data\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:27.243925 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.243900    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" (UID: "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:26:27.244010 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.243898    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" (UID: "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 19:26:27.244163 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.244141    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-kube-api-access-qtrjh" (OuterVolumeSpecName: "kube-api-access-qtrjh") pod "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" (UID: "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec"). InnerVolumeSpecName "kube-api-access-qtrjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:26:27.244210 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.244173    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" (UID: "6d40c5ac-89c4-46b4-bb40-c7b44f1efcec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:26:27.343372 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.343344    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtrjh\" (UniqueName: \"kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-kube-api-access-qtrjh\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:27.343372 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.343368    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-envoy\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:27.343372 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.343378    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-podinfo\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:27.343564 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.343386    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec-istio-token\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:27.832695 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.832660    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerID="188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d" exitCode=0
Aug 04 19:26:27.832860 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.832703    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"
Aug 04 19:26:27.832860 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.832744    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" event={"ID":"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec","Type":"ContainerDied","Data":"188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d"}
Aug 04 19:26:27.832860 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.832788    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f" event={"ID":"6d40c5ac-89c4-46b4-bb40-c7b44f1efcec","Type":"ContainerDied","Data":"c5b5b2f7e4411085240628b17839ba9f5a13d5c0d97e1f7edb6ec16af54f6508"}
Aug 04 19:26:27.832860 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.832811    2578 scope.go:117] "RemoveContainer" containerID="188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d"
Aug 04 19:26:27.842268 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.842252    2578 scope.go:117] "RemoveContainer" containerID="be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22"
Aug 04 19:26:27.849875 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.849860    2578 scope.go:117] "RemoveContainer" containerID="188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d"
Aug 04 19:26:27.850098 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:26:27.850079    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d\": container with ID starting with 188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d not found: ID does not exist" containerID="188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d"
Aug 04 19:26:27.850142 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.850105    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d"} err="failed to get container status \"188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d\": rpc error: code = NotFound desc = could not find container \"188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d\": container with ID starting with 188f885c2fff59ebfe51a7b3bd95cb05952b32b0aebffcb6ae00a657a4aa214d not found: ID does not exist"
Aug 04 19:26:27.850142 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.850120    2578 scope.go:117] "RemoveContainer" containerID="be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22"
Aug 04 19:26:27.850335 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:26:27.850321    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22\": container with ID starting with be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22 not found: ID does not exist" containerID="be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22"
Aug 04 19:26:27.850370 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.850337    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22"} err="failed to get container status \"be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22\": rpc error: code = NotFound desc = could not find container \"be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22\": container with ID starting with be2f6e8f3a68cf5a4f1b8de25f2bfc245fa7a944201a7a0822d3fcd9cb750a22 not found: ID does not exist"
Aug 04 19:26:27.854345 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.854325    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"]
Aug 04 19:26:27.860652 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:27.860629    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-f18332-openshift-default-7b4578b658-hfc8f"]
Aug 04 19:26:28.802199 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:28.802167    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" path="/var/lib/kubelet/pods/6d40c5ac-89c4-46b4-bb40-c7b44f1efcec/volumes"
Aug 04 19:26:32.808882 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:32.808850    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:32.855100 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:32.855073    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"]
Aug 04 19:26:32.855293 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:32.855269    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp" podUID="ecfb1071-9804-4a80-a66e-489a61bc6938" containerName="payload-pre-processing" containerID="cri-o://1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae" gracePeriod=30
Aug 04 19:26:32.889566 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:32.889543    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:32.993160 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:32.993129    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:33.091306 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.091219    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:33.121279 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.121252    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:26:33.189111 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.189079    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc9vn\" (UniqueName: \"kubernetes.io/projected/ecfb1071-9804-4a80-a66e-489a61bc6938-kube-api-access-cc9vn\") pod \"ecfb1071-9804-4a80-a66e-489a61bc6938\" (UID: \"ecfb1071-9804-4a80-a66e-489a61bc6938\") "
Aug 04 19:26:33.189297 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.189152    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfb1071-9804-4a80-a66e-489a61bc6938-plugins-config-volume\") pod \"ecfb1071-9804-4a80-a66e-489a61bc6938\" (UID: \"ecfb1071-9804-4a80-a66e-489a61bc6938\") "
Aug 04 19:26:33.189553 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.189525    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecfb1071-9804-4a80-a66e-489a61bc6938-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecfb1071-9804-4a80-a66e-489a61bc6938" (UID: "ecfb1071-9804-4a80-a66e-489a61bc6938"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:26:33.191293 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.191264    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecfb1071-9804-4a80-a66e-489a61bc6938-kube-api-access-cc9vn" (OuterVolumeSpecName: "kube-api-access-cc9vn") pod "ecfb1071-9804-4a80-a66e-489a61bc6938" (UID: "ecfb1071-9804-4a80-a66e-489a61bc6938"). InnerVolumeSpecName "kube-api-access-cc9vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:26:33.290074 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.290039    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecfb1071-9804-4a80-a66e-489a61bc6938-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:33.290074 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.290068    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc9vn\" (UniqueName: \"kubernetes.io/projected/ecfb1071-9804-4a80-a66e-489a61bc6938-kube-api-access-cc9vn\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:33.439614 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.439505    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:33.862615 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.862556    2578 generic.go:356] "Generic (PLEG): container finished" podID="ecfb1071-9804-4a80-a66e-489a61bc6938" containerID="1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae" exitCode=0
Aug 04 19:26:33.863091 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.862623    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"
Aug 04 19:26:33.863091 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.862648    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp" event={"ID":"ecfb1071-9804-4a80-a66e-489a61bc6938","Type":"ContainerDied","Data":"1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae"}
Aug 04 19:26:33.863091 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.862703    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp" event={"ID":"ecfb1071-9804-4a80-a66e-489a61bc6938","Type":"ContainerDied","Data":"5eb53540b0cdfe3dd5bdb7f560ad807ae606d38ad0b5aea7da8e964e222b1b58"}
Aug 04 19:26:33.863091 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.862727    2578 scope.go:117] "RemoveContainer" containerID="1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae"
Aug 04 19:26:33.871019 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.870997    2578 scope.go:117] "RemoveContainer" containerID="1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae"
Aug 04 19:26:33.871261 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:26:33.871244    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae\": container with ID starting with 1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae not found: ID does not exist" containerID="1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae"
Aug 04 19:26:33.871315 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.871270    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae"} err="failed to get container status \"1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae\": rpc error: code = NotFound desc = could not find container \"1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae\": container with ID starting with 1b98d95abed734b5a9c5a4b61e8819d7414a2aee6c09303c96549050b4a5d1ae not found: ID does not exist"
Aug 04 19:26:33.884918 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.884895    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"]
Aug 04 19:26:33.889222 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:33.889203    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-1d2c46-7f89ff75cb-j6rgp"]
Aug 04 19:26:34.802108 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:34.802074    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecfb1071-9804-4a80-a66e-489a61bc6938" path="/var/lib/kubelet/pods/ecfb1071-9804-4a80-a66e-489a61bc6938/volumes"
Aug 04 19:26:39.611263 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:39.611231    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:39.711749 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:39.711701    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:39.876347 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:39.876264    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"]
Aug 04 19:26:39.876491 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:39.876467    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy" containerID="cri-o://f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613" gracePeriod=30
Aug 04 19:26:39.905526 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:39.905498    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:40.072296 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:40.072269    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:40.275413 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:40.275332    2578 patch_prober.go:28] interesting pod/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 19:26:40.275413 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:40.275399    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 19:26:45.130917 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.130888    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:26:45.203619 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.203525    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-data\") pod \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") "
Aug 04 19:26:45.203619 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.203563    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-envoy\") pod \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") "
Aug 04 19:26:45.203825 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.203633    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldcgc\" (UniqueName: \"kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-kube-api-access-ldcgc\") pod \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") "
Aug 04 19:26:45.203825 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.203657    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-socket\") pod \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") "
Aug 04 19:26:45.203825 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.203685    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-token\") pod \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") "
Aug 04 19:26:45.203825 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.203736    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-certs\") pod \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") "
Aug 04 19:26:45.203825 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.203763    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istiod-ca-cert\") pod \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") "
Aug 04 19:26:45.203825 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.203766    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ed58469-64e8-4a86-b99e-7096e2e5f42b" (UID: "9ed58469-64e8-4a86-b99e-7096e2e5f42b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:26:45.203825 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.203800    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-podinfo\") pod \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") "
Aug 04 19:26:45.204169 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.203827    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-credential-socket\") pod \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\" (UID: \"9ed58469-64e8-4a86-b99e-7096e2e5f42b\") "
Aug 04 19:26:45.204169 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.203983    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ed58469-64e8-4a86-b99e-7096e2e5f42b" (UID: "9ed58469-64e8-4a86-b99e-7096e2e5f42b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:26:45.204169 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.204126    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-data\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:45.204169 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.204146    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-certs\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:45.204361 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.204209    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ed58469-64e8-4a86-b99e-7096e2e5f42b" (UID: "9ed58469-64e8-4a86-b99e-7096e2e5f42b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:26:45.204361 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.204245    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ed58469-64e8-4a86-b99e-7096e2e5f42b" (UID: "9ed58469-64e8-4a86-b99e-7096e2e5f42b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:26:45.204361 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.204296    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ed58469-64e8-4a86-b99e-7096e2e5f42b" (UID: "9ed58469-64e8-4a86-b99e-7096e2e5f42b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:26:45.205802 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.205779    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-kube-api-access-ldcgc" (OuterVolumeSpecName: "kube-api-access-ldcgc") pod "9ed58469-64e8-4a86-b99e-7096e2e5f42b" (UID: "9ed58469-64e8-4a86-b99e-7096e2e5f42b"). InnerVolumeSpecName "kube-api-access-ldcgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:26:45.205904 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.205824    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ed58469-64e8-4a86-b99e-7096e2e5f42b" (UID: "9ed58469-64e8-4a86-b99e-7096e2e5f42b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 19:26:45.205976 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.205945    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ed58469-64e8-4a86-b99e-7096e2e5f42b" (UID: "9ed58469-64e8-4a86-b99e-7096e2e5f42b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 19:26:45.206042 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.206024    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ed58469-64e8-4a86-b99e-7096e2e5f42b" (UID: "9ed58469-64e8-4a86-b99e-7096e2e5f42b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:26:45.305356 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.305322    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldcgc\" (UniqueName: \"kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-kube-api-access-ldcgc\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:45.305356 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.305353    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-workload-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:45.305356 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.305366    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-token\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:45.305609 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.305378    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istiod-ca-cert\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:45.305609 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.305389    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-podinfo\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:45.305609 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.305400    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-credential-socket\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:45.305609 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.305410    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ed58469-64e8-4a86-b99e-7096e2e5f42b-istio-envoy\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:45.921311 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.921273    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerID="f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613" exitCode=0
Aug 04 19:26:45.921483 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.921318    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"
Aug 04 19:26:45.921483 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.921350    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" event={"ID":"9ed58469-64e8-4a86-b99e-7096e2e5f42b","Type":"ContainerDied","Data":"f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613"}
Aug 04 19:26:45.921483 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.921402    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd" event={"ID":"9ed58469-64e8-4a86-b99e-7096e2e5f42b","Type":"ContainerDied","Data":"74316bfeeb47c3456cf108fb821d32143c0ed7e5a2acef23fa6e6c287073059d"}
Aug 04 19:26:45.921483 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.921426    2578 scope.go:117] "RemoveContainer" containerID="f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613"
Aug 04 19:26:45.930315 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.930298    2578 scope.go:117] "RemoveContainer" containerID="0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a"
Aug 04 19:26:45.937754 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.937733    2578 scope.go:117] "RemoveContainer" containerID="f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613"
Aug 04 19:26:45.938009 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:26:45.937990    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613\": container with ID starting with f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613 not found: ID does not exist" containerID="f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613"
Aug 04 19:26:45.938071 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.938018    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613"} err="failed to get container status \"f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613\": rpc error: code = NotFound desc = could not find container \"f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613\": container with ID starting with f64feaffa94baab741510d76f554c572725128c0d909f4293a6641a9125e8613 not found: ID does not exist"
Aug 04 19:26:45.938071 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.938036    2578 scope.go:117] "RemoveContainer" containerID="0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a"
Aug 04 19:26:45.938244 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:26:45.938230    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a\": container with ID starting with 0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a not found: ID does not exist" containerID="0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a"
Aug 04 19:26:45.938278 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.938248    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a"} err="failed to get container status \"0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a\": rpc error: code = NotFound desc = could not find container \"0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a\": container with ID starting with 0660e78eb37df87f5292845a2c71468bbf19cf33dfe88d41f776566bd436da7a not found: ID does not exist"
Aug 04 19:26:45.943564 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.943543    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"]
Aug 04 19:26:45.948878 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:45.948859    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-1d2c46-openshift-default-78c5c64956-fvsjd"]
Aug 04 19:26:46.802464 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:46.802431    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" path="/var/lib/kubelet/pods/9ed58469-64e8-4a86-b99e-7096e2e5f42b/volumes"
Aug 04 19:26:50.803825 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:50.803792    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:50.841887 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:50.841861    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"]
Aug 04 19:26:50.842102 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:50.842076    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm" podUID="b02c38d2-440f-4b44-ae0c-151f12158b77" containerName="payload-processing" containerID="cri-o://7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681" gracePeriod=30
Aug 04 19:26:50.848040 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:50.848016    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"]
Aug 04 19:26:50.848241 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:50.848218    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd" podUID="38682654-a0a1-432d-b404-6f97bd365452" containerName="payload-pre-processing" containerID="cri-o://8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269" gracePeriod=30
Aug 04 19:26:50.884524 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:50.884475    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:50.952441 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:50.952414    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:51.026903 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.026793    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:51.129881 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.129857    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:26:51.132347 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.132331    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:26:51.256652 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.256621    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b02c38d2-440f-4b44-ae0c-151f12158b77-plugins-config-volume\") pod \"b02c38d2-440f-4b44-ae0c-151f12158b77\" (UID: \"b02c38d2-440f-4b44-ae0c-151f12158b77\") "
Aug 04 19:26:51.256652 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.256655    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64zpm\" (UniqueName: \"kubernetes.io/projected/b02c38d2-440f-4b44-ae0c-151f12158b77-kube-api-access-64zpm\") pod \"b02c38d2-440f-4b44-ae0c-151f12158b77\" (UID: \"b02c38d2-440f-4b44-ae0c-151f12158b77\") "
Aug 04 19:26:51.256899 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.256696    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jljgn\" (UniqueName: \"kubernetes.io/projected/38682654-a0a1-432d-b404-6f97bd365452-kube-api-access-jljgn\") pod \"38682654-a0a1-432d-b404-6f97bd365452\" (UID: \"38682654-a0a1-432d-b404-6f97bd365452\") "
Aug 04 19:26:51.256899 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.256771    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38682654-a0a1-432d-b404-6f97bd365452-plugins-config-volume\") pod \"38682654-a0a1-432d-b404-6f97bd365452\" (UID: \"38682654-a0a1-432d-b404-6f97bd365452\") "
Aug 04 19:26:51.257061 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.257039    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38682654-a0a1-432d-b404-6f97bd365452-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "38682654-a0a1-432d-b404-6f97bd365452" (UID: "38682654-a0a1-432d-b404-6f97bd365452"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:26:51.257109 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.257047    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b02c38d2-440f-4b44-ae0c-151f12158b77-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b02c38d2-440f-4b44-ae0c-151f12158b77" (UID: "b02c38d2-440f-4b44-ae0c-151f12158b77"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 19:26:51.258873 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.258852    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38682654-a0a1-432d-b404-6f97bd365452-kube-api-access-jljgn" (OuterVolumeSpecName: "kube-api-access-jljgn") pod "38682654-a0a1-432d-b404-6f97bd365452" (UID: "38682654-a0a1-432d-b404-6f97bd365452"). InnerVolumeSpecName "kube-api-access-jljgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:26:51.258958 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.258874    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b02c38d2-440f-4b44-ae0c-151f12158b77-kube-api-access-64zpm" (OuterVolumeSpecName: "kube-api-access-64zpm") pod "b02c38d2-440f-4b44-ae0c-151f12158b77" (UID: "b02c38d2-440f-4b44-ae0c-151f12158b77"). InnerVolumeSpecName "kube-api-access-64zpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 19:26:51.358067 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.357994    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jljgn\" (UniqueName: \"kubernetes.io/projected/38682654-a0a1-432d-b404-6f97bd365452-kube-api-access-jljgn\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:51.358067 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.358019    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38682654-a0a1-432d-b404-6f97bd365452-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:51.358067 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.358029    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b02c38d2-440f-4b44-ae0c-151f12158b77-plugins-config-volume\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:51.358067 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.358037    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64zpm\" (UniqueName: \"kubernetes.io/projected/b02c38d2-440f-4b44-ae0c-151f12158b77-kube-api-access-64zpm\") on node \"ip-10-0-128-139.ec2.internal\" DevicePath \"\""
Aug 04 19:26:51.376915 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.376889    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:51.952561 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.952530    2578 generic.go:356] "Generic (PLEG): container finished" podID="b02c38d2-440f-4b44-ae0c-151f12158b77" containerID="7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681" exitCode=0
Aug 04 19:26:51.952561 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.952567    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"
Aug 04 19:26:51.953037 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.952611    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm" event={"ID":"b02c38d2-440f-4b44-ae0c-151f12158b77","Type":"ContainerDied","Data":"7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681"}
Aug 04 19:26:51.953037 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.952661    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm" event={"ID":"b02c38d2-440f-4b44-ae0c-151f12158b77","Type":"ContainerDied","Data":"f4ad329923c11f89170a57a96d4af702413f5ca612727ed8f7987f753d7152fb"}
Aug 04 19:26:51.953037 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.952691    2578 scope.go:117] "RemoveContainer" containerID="7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681"
Aug 04 19:26:51.953961 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.953941    2578 generic.go:356] "Generic (PLEG): container finished" podID="38682654-a0a1-432d-b404-6f97bd365452" containerID="8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269" exitCode=0
Aug 04 19:26:51.954034 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.953971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd" event={"ID":"38682654-a0a1-432d-b404-6f97bd365452","Type":"ContainerDied","Data":"8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269"}
Aug 04 19:26:51.954034 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.953995    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"
Aug 04 19:26:51.954034 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.954021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd" event={"ID":"38682654-a0a1-432d-b404-6f97bd365452","Type":"ContainerDied","Data":"606918a859bc27dd0cccbebafeb74191900af789818c351bc5cf14b48e9a046a"}
Aug 04 19:26:51.963306 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.963284    2578 scope.go:117] "RemoveContainer" containerID="7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681"
Aug 04 19:26:51.963956 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:26:51.963652    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681\": container with ID starting with 7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681 not found: ID does not exist" containerID="7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681"
Aug 04 19:26:51.963956 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.963684    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681"} err="failed to get container status \"7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681\": rpc error: code = NotFound desc = could not find container \"7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681\": container with ID starting with 7a1a0c05ef4380cd8db2c9a674f70c4c0e19865a7a79d5bf11000895301e2681 not found: ID does not exist"
Aug 04 19:26:51.963956 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.963706    2578 scope.go:117] "RemoveContainer" containerID="8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269"
Aug 04 19:26:51.972217 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.972201    2578 scope.go:117] "RemoveContainer" containerID="8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269"
Aug 04 19:26:51.972455 ip-10-0-128-139 kubenswrapper[2578]: E0804 19:26:51.972439    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269\": container with ID starting with 8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269 not found: ID does not exist" containerID="8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269"
Aug 04 19:26:51.972490 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.972463    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269"} err="failed to get container status \"8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269\": rpc error: code = NotFound desc = could not find container \"8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269\": container with ID starting with 8dab0692fa822076d9c0c1a960db203090daf7157f68415240f731cf80be4269 not found: ID does not exist"
Aug 04 19:26:51.982084 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.982063    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"]
Aug 04 19:26:51.987437 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.987418    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-1a00f6-7d975c47d4-45tmd"]
Aug 04 19:26:51.999755 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:51.999731    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"]
Aug 04 19:26:52.016680 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:52.016654    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-1a00f6-57c547d686-bwrwm"]
Aug 04 19:26:52.802102 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:52.802071    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38682654-a0a1-432d-b404-6f97bd365452" path="/var/lib/kubelet/pods/38682654-a0a1-432d-b404-6f97bd365452/volumes"
Aug 04 19:26:52.802430 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:52.802417    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b02c38d2-440f-4b44-ae0c-151f12158b77" path="/var/lib/kubelet/pods/b02c38d2-440f-4b44-ae0c-151f12158b77/volumes"
Aug 04 19:26:57.861120 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:57.861090    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:57.968392 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:57.968355    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:58.152855 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:58.152760    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:26:58.378740 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:26:58.378699    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:27:03.941241 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:03.941210    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-nhls4_1f34e173-c23c-401d-ac4d-1606b9139768/manager/0.log"
Aug 04 19:27:04.062008 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:04.061985    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pd99f_7c6ca52c-4af7-4bbe-a80b-23c5d1b59064/manager/0.log"
Aug 04 19:27:04.289744 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:04.289721    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-v6srd_23a1c5ee-79b3-4ac8-8e98-419be848af7d/server/0.log"
Aug 04 19:27:04.403936 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:04.403908    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-55rcx_b7a08462-4239-49ba-836c-363b6fa441bb/manager/0.log"
Aug 04 19:27:04.634310 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:04.634249    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5556d6485b-cs9tl_1d8e147b-4158-41ec-87da-f1c21ce06121/copy-manifests/0.log"
Aug 04 19:27:04.641560 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:04.641537    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5556d6485b-cs9tl_1d8e147b-4158-41ec-87da-f1c21ce06121/manager/0.log"
Aug 04 19:27:06.407270 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:06.407243    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-9d264_45e54eea-b7e4-4ea8-b482-41a6e7698942/manager/0.log"
Aug 04 19:27:06.510304 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:06.510279    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-c6dbz_bbf8ae72-cbf7-4d33-a481-5c0d6ed2ef7d/manager/0.log"
Aug 04 19:27:06.727773 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:06.727697    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-2v4vs_2b1a32d3-57c0-4a09-9ee1-733b0350be45/manager/0.log"
Aug 04 19:27:06.941218 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:06.941188    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-7v927_7d9a08d1-a47c-4b12-8039-4f482b6d4e5f/manager/0.log"
Aug 04 19:27:07.267878 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:07.267851    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht_44c62d1c-6255-4540-bb8a-ae34288d1b0f/istio-proxy/0.log"
Aug 04 19:27:07.601207 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:07.601179    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f654896c-bgwnz_7ac5c5b2-4c34-4398-aa0e-d1907eae44e2/kube-auth-proxy/0.log"
Aug 04 19:27:07.718642 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:07.718615    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xfrfc_ecb3535a-f51b-4b45-b3f8-03479217df75/istio-proxy/1.log"
Aug 04 19:27:07.832466 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:07.832440    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-lt4zt_10355037-0234-4030-a93d-eccdcba6252b/payload-pre-processing/0.log"
Aug 04 19:27:08.480611 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:08.480562    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vqp4r_50921b6e-45e6-43cb-8b39-7a66c75cea95/main/0.log"
Aug 04 19:27:08.590103 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:08.590078    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-bxxlc_7c19e1e5-851a-4399-826d-90bed88da3aa/main/0.log"
Aug 04 19:27:08.916429 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:08.916401    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-cgllg_1a98801e-320a-48fd-bde6-fa151e154182/main/0.log"
Aug 04 19:27:14.681308 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:14.681276    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:27:14.891052 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:14.886310    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log" limit=1024
Aug 04 19:27:17.969920 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:17.969838    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h2rx8_d2ecaa08-30d0-47a3-b32f-50486a0462e2/global-pull-secret-syncer/0.log"
Aug 04 19:27:18.125195 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:18.125166    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hwjp9_05646d91-5551-4043-8542-b01200ff9771/konnectivity-agent/0.log"
Aug 04 19:27:18.261502 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:18.261430    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-139.ec2.internal_7d0a2e4e61221718b5319001dcdbfbbc/haproxy/0.log"
Aug 04 19:27:23.069761 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:23.069729    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-9d264_45e54eea-b7e4-4ea8-b482-41a6e7698942/manager/0.log"
Aug 04 19:27:23.103474 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:23.103453    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-c6dbz_bbf8ae72-cbf7-4d33-a481-5c0d6ed2ef7d/manager/0.log"
Aug 04 19:27:23.229969 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:23.229943    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-2v4vs_2b1a32d3-57c0-4a09-9ee1-733b0350be45/manager/0.log"
Aug 04 19:27:23.356861 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:23.356775    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-7v927_7d9a08d1-a47c-4b12-8039-4f482b6d4e5f/manager/0.log"
Aug 04 19:27:25.338144 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:25.338111    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-45xst_9006431f-fe11-4a12-8b69-988b25b48dd8/node-exporter/0.log"
Aug 04 19:27:25.377852 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:25.377792    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-45xst_9006431f-fe11-4a12-8b69-988b25b48dd8/kube-rbac-proxy/0.log"
Aug 04 19:27:25.408724 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:25.408700    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-45xst_9006431f-fe11-4a12-8b69-988b25b48dd8/init-textfile/0.log"
Aug 04 19:27:28.582295 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582265    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"]
Aug 04 19:27:28.582681 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582607    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerName="istio-proxy"
Aug 04 19:27:28.582681 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582621    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerName="istio-proxy"
Aug 04 19:27:28.582681 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582635    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy"
Aug 04 19:27:28.582681 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582641    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy"
Aug 04 19:27:28.582681 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582650    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy"
Aug 04 19:27:28.582681 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582655    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy"
Aug 04 19:27:28.582681 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582662    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy"
Aug 04 19:27:28.582681 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582668    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy"
Aug 04 19:27:28.582681 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582677    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecfb1071-9804-4a80-a66e-489a61bc6938" containerName="payload-pre-processing"
Aug 04 19:27:28.582681 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582685    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecfb1071-9804-4a80-a66e-489a61bc6938" containerName="payload-pre-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582695    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b02c38d2-440f-4b44-ae0c-151f12158b77" containerName="payload-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582700    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b02c38d2-440f-4b44-ae0c-151f12158b77" containerName="payload-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582708    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcbe73c7-ebc2-48e2-95c9-81b49405cdc6" containerName="payload-pre-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582713    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcbe73c7-ebc2-48e2-95c9-81b49405cdc6" containerName="payload-pre-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582720    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38682654-a0a1-432d-b404-6f97bd365452" containerName="payload-pre-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582725    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="38682654-a0a1-432d-b404-6f97bd365452" containerName="payload-pre-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582730    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cfe8865-0d4a-46a4-a38a-b565e1f83f33" containerName="payload-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582735    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cfe8865-0d4a-46a4-a38a-b565e1f83f33" containerName="payload-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582742    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582747    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582797    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed58469-64e8-4a86-b99e-7096e2e5f42b" containerName="istio-proxy"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582805    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cfe8865-0d4a-46a4-a38a-b565e1f83f33" containerName="payload-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582811    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582818    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcbe73c7-ebc2-48e2-95c9-81b49405cdc6" containerName="payload-pre-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582823    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c4764ff-12b3-4522-a890-039ce1fe1f8d" containerName="istio-proxy"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582828    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="38682654-a0a1-432d-b404-6f97bd365452" containerName="payload-pre-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582834    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerName="istio-proxy"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582842    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecfb1071-9804-4a80-a66e-489a61bc6938" containerName="payload-pre-processing"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582864    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d40c5ac-89c4-46b4-bb40-c7b44f1efcec" containerName="istio-proxy"
Aug 04 19:27:28.582985 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.582871    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b02c38d2-440f-4b44-ae0c-151f12158b77" containerName="payload-processing"
Aug 04 19:27:28.586011 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.585994    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.588608 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.588586    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-szqsg\"/\"kube-root-ca.crt\""
Aug 04 19:27:28.588608 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.588603    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-szqsg\"/\"openshift-service-ca.crt\""
Aug 04 19:27:28.588756 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.588620    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-szqsg\"/\"default-dockercfg-7dnkb\""
Aug 04 19:27:28.607976 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.607957    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"]
Aug 04 19:27:28.670075 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.670044    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-proc\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.670200 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.670080    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-sys\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.670200 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.670116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8j5b\" (UniqueName: \"kubernetes.io/projected/fba9d361-8e4f-431c-b355-004c60141b66-kube-api-access-k8j5b\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.670200 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.670166    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-podres\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.670343 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.670249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-lib-modules\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.771646 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.771610    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-sys\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.771794 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.771658    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8j5b\" (UniqueName: \"kubernetes.io/projected/fba9d361-8e4f-431c-b355-004c60141b66-kube-api-access-k8j5b\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.771794 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.771679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-podres\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.771794 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.771745    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-sys\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.771924 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.771794    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-podres\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.771924 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.771826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-lib-modules\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.771924 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.771883    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-proc\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.772060 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.771943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-proc\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.772060 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.772008    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fba9d361-8e4f-431c-b355-004c60141b66-lib-modules\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.799751 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.799728    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8j5b\" (UniqueName: \"kubernetes.io/projected/fba9d361-8e4f-431c-b355-004c60141b66-kube-api-access-k8j5b\") pod \"perf-node-gather-daemonset-d58zr\" (UID: \"fba9d361-8e4f-431c-b355-004c60141b66\") " pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:28.895676 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:28.895618    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:29.044056 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:29.044033    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"]
Aug 04 19:27:29.045177 ip-10-0-128-139 kubenswrapper[2578]: W0804 19:27:29.045152    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfba9d361_8e4f_431c_b355_004c60141b66.slice/crio-2f0cb8c42ff701c1b3455bd4afd5aad4f51dc15f0370a6112e0804e2caba8710 WatchSource:0}: Error finding container 2f0cb8c42ff701c1b3455bd4afd5aad4f51dc15f0370a6112e0804e2caba8710: Status 404 returned error can't find the container with id 2f0cb8c42ff701c1b3455bd4afd5aad4f51dc15f0370a6112e0804e2caba8710
Aug 04 19:27:29.130341 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:29.130313    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr" event={"ID":"fba9d361-8e4f-431c-b355-004c60141b66","Type":"ContainerStarted","Data":"1b923ac315ae42d6ac9f39d1d7835430c604eee99fe23e1eb7e784324763f489"}
Aug 04 19:27:29.130454 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:29.130390    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr" event={"ID":"fba9d361-8e4f-431c-b355-004c60141b66","Type":"ContainerStarted","Data":"2f0cb8c42ff701c1b3455bd4afd5aad4f51dc15f0370a6112e0804e2caba8710"}
Aug 04 19:27:29.130454 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:29.130444    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:29.179589 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:29.179505    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr" podStartSLOduration=1.17949268 podStartE2EDuration="1.17949268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 19:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 19:27:29.161699829 +0000 UTC m=+3591.028614931" watchObservedRunningTime="2026-08-04 19:27:29.17949268 +0000 UTC m=+3591.046407779"
Aug 04 19:27:31.544278 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:31.544206    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jwfsp_1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11/dns/0.log"
Aug 04 19:27:31.578920 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:31.578897    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jwfsp_1d4b4d2c-fbc2-4d91-aebd-a0ecf05dfe11/kube-rbac-proxy/0.log"
Aug 04 19:27:31.712792 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:31.712766    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8jqq2_da06502a-2382-4c5c-aa50-06912a1cadaf/dns-node-resolver/0.log"
Aug 04 19:27:32.489249 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:32.489223    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9g8kd_274adda4-c5aa-448e-baf7-d4b48140929c/node-ca/0.log"
Aug 04 19:27:33.666120 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:33.666093    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht_44c62d1c-6255-4540-bb8a-ae34288d1b0f/istio-proxy/0.log"
Aug 04 19:27:33.912494 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:33.912466    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f654896c-bgwnz_7ac5c5b2-4c34-4398-aa0e-d1907eae44e2/kube-auth-proxy/0.log"
Aug 04 19:27:33.945767 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:33.945699    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xfrfc_ecb3535a-f51b-4b45-b3f8-03479217df75/istio-proxy/0.log"
Aug 04 19:27:33.951280 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:33.951261    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xfrfc_ecb3535a-f51b-4b45-b3f8-03479217df75/istio-proxy/1.log"
Aug 04 19:27:33.982387 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:33.982355    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-lt4zt_10355037-0234-4030-a93d-eccdcba6252b/payload-pre-processing/0.log"
Aug 04 19:27:35.006415 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:35.006386    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbxls_61245fc9-3d68-4d5b-a57e-3f3c3bc9b6b6/serve-healthcheck-canary/0.log"
Aug 04 19:27:35.144200 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:35.144178    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-szqsg/perf-node-gather-daemonset-d58zr"
Aug 04 19:27:35.789934 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:35.789905    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgvv8_65dfd213-63dd-44d3-9803-9b33b337ac19/kube-rbac-proxy/0.log"
Aug 04 19:27:35.814935 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:35.814907    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgvv8_65dfd213-63dd-44d3-9803-9b33b337ac19/exporter/0.log"
Aug 04 19:27:35.841816 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:35.841794    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgvv8_65dfd213-63dd-44d3-9803-9b33b337ac19/extractor/0.log"
Aug 04 19:27:38.455899 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:38.455870    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-nhls4_1f34e173-c23c-401d-ac4d-1606b9139768/manager/0.log"
Aug 04 19:27:38.708666 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:38.708569    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pd99f_7c6ca52c-4af7-4bbe-a80b-23c5d1b59064/manager/0.log"
Aug 04 19:27:38.975244 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:38.975177    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-v6srd_23a1c5ee-79b3-4ac8-8e98-419be848af7d/server/0.log"
Aug 04 19:27:39.100097 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:39.100076    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-55rcx_b7a08462-4239-49ba-836c-363b6fa441bb/manager/0.log"
Aug 04 19:27:39.293399 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:39.293373    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5556d6485b-cs9tl_1d8e147b-4158-41ec-87da-f1c21ce06121/manager/0.log"
Aug 04 19:27:39.337864 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:39.337842    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5556d6485b-cs9tl_1d8e147b-4158-41ec-87da-f1c21ce06121/copy-manifests/0.log"
Aug 04 19:27:39.705339 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:39.705265    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht_44c62d1c-6255-4540-bb8a-ae34288d1b0f/istio-proxy/0.log"
Aug 04 19:27:39.942540 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:39.942513    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f654896c-bgwnz_7ac5c5b2-4c34-4398-aa0e-d1907eae44e2/kube-auth-proxy/0.log"
Aug 04 19:27:39.985313 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:39.985219    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xfrfc_ecb3535a-f51b-4b45-b3f8-03479217df75/istio-proxy/0.log"
Aug 04 19:27:39.991350 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:39.991330    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xfrfc_ecb3535a-f51b-4b45-b3f8-03479217df75/istio-proxy/1.log"
Aug 04 19:27:40.018911 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:40.018892    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-lt4zt_10355037-0234-4030-a93d-eccdcba6252b/payload-pre-processing/0.log"
Aug 04 19:27:41.093546 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:41.093517    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-nvfh9_3b57ce7d-5c95-4647-a1d9-e08cee19eaeb/openshift-lws-operator/0.log"
Aug 04 19:27:41.742913 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:41.742884    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-b82ht_44c62d1c-6255-4540-bb8a-ae34288d1b0f/istio-proxy/0.log"
Aug 04 19:27:41.942506 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:41.942484    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f654896c-bgwnz_7ac5c5b2-4c34-4398-aa0e-d1907eae44e2/kube-auth-proxy/0.log"
Aug 04 19:27:41.965918 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:41.965887    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xfrfc_ecb3535a-f51b-4b45-b3f8-03479217df75/istio-proxy/0.log"
Aug 04 19:27:41.971129 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:41.971111    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xfrfc_ecb3535a-f51b-4b45-b3f8-03479217df75/istio-proxy/1.log"
Aug 04 19:27:42.002609 ip-10-0-128-139 kubenswrapper[2578]: I0804 19:27:42.002545    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-lt4zt_10355037-0234-4030-a93d-eccdcba6252b/payload-pre-processing/0.log"