Aug 04 03:09:41.213441 ip-10-0-134-110 systemd[1]: Starting Kubernetes Kubelet...
Aug 04 03:09:42.981583 ip-10-0-134-110 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 03:09:42.981583 ip-10-0-134-110 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 04 03:09:42.981583 ip-10-0-134-110 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 03:09:42.981583 ip-10-0-134-110 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 04 03:09:42.981583 ip-10-0-134-110 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 03:09:42.982625 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.982519    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 04 03:09:42.988831 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988809    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 03:09:42.988831 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988830    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 03:09:42.988831 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988834    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988838    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988842    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988844    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988847    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988851    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988854    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988856    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988859    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988862    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988864    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988867    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988870    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988873    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988875    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988880    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988885    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988888    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988891    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 03:09:42.988930 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988895    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988898    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988900    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988903    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988906    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988908    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988911    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988914    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988916    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988919    2559 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988921    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988924    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988927    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988931    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988936    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988939    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988941    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988944    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988947    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 03:09:42.989421 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988950    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988953    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988956    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988959    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988961    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988964    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988966    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988969    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988972    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988974    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988978    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988980    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988983    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988986    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988988    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988991    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988995    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.988999    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989002    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 03:09:42.989901 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989005    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989008    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989010    2559 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989013    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989016    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989019    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989022    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989025    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989029    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989032    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989035    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989037    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989040    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989043    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989045    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989048    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989050    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989053    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989055    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989058    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 03:09:42.990365 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989061    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989063    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989066    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989068    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989070    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989073    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989076    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989078    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989081    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989083    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989087    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989089    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989092    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989095    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989097    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989100    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989102    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989105    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989107    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989111    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 03:09:42.990892 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989853    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989860    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989864    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989866    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989870    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989879    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989882    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989885    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989888    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989890    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989893    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989896    2559 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989899    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989901    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989904    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989907    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989911    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989915    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989919    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 03:09:42.991453 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989922    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989924    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989927    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989931    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989934    2559 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989937    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989942    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989945    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989948    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989950    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989953    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989956    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989959    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989962    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989965    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989967    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989969    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989972    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989974    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989978    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 03:09:42.992043 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989980    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989983    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989985    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989988    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989990    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989993    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989995    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.989998    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990000    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990003    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990005    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990008    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990010    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990012    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990015    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990017    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990020    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990022    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990025    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990027    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 03:09:42.992618 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990030    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990032    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990035    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990038    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990042    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990045    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990047    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990050    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990052    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990056    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990058    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990061    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990064    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990066    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990069    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990072    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990074    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990077    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990079    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990082    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 03:09:42.993186 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990085    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990087    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990089    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990092    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990094    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990096    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990099    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990101    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990104    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990107    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990109    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990112    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990114    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990117    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990120    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990122    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990125    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990128    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990130    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990133    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 03:09:42.993758 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990574    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990581    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990583    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990586    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990589    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990592    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990595    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990597    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990600    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990603    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990605    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990608    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990610    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990613    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990615    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990618    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990620    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990623    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990626    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990628    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 03:09:42.994310 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990631    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990634    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990637    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990640    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990642    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990645    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990647    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990650    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990653    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990657    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990661    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990664    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990667    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990670    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990673    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990675    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990678    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990682    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990685    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990687    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 03:09:42.994972 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990690    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990692    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990695    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990697    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990700    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990702    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990705    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990708    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990710    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990713    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990715    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990719    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990722    2559 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990725    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990728    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990731    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990734    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990737    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990741    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990743    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 03:09:42.995504 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990746    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990748    2559 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990751    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990753    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990756    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990758    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990760    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990763    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990765    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990768    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990771    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990773    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990776    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990778    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990781    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990784    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990787    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990790    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990793    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990796    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 03:09:42.996023 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990799    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990801    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990804    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990806    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990810    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990813    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990815    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990818    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990821    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990823    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990826    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990829    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990831    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990834    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990836    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990839    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990842    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990845    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.990847    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990933    2559 flags.go:64] FLAG: --address="0.0.0.0"
Aug 04 03:09:42.996524 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990942    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990949    2559 flags.go:64] FLAG: --anonymous-auth="true"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990954    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990962    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990966    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990970    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990975    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990978    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990981    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990985    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990988    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990991    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990994    2559 flags.go:64] FLAG: --cgroup-root=""
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.990997    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991000    2559 flags.go:64] FLAG: --client-ca-file=""
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991004    2559 flags.go:64] FLAG: --cloud-provider="external"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991007    2559 flags.go:64] FLAG: --cluster-dns="[]"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991011    2559 flags.go:64] FLAG: --cluster-domain=""
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991014    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991017    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991020    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991023    2559 flags.go:64] FLAG: --container-log-max-files="5"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991027    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 04 03:09:42.997025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991030    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991033    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991037    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991040    2559 flags.go:64] FLAG: --contention-profiling="false"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991043    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991046    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991049    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991052    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991056    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991059    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991062    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991065    2559 flags.go:64] FLAG: --enable-load-reader="false"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991068    2559 flags.go:64] FLAG: --enable-server="true"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991075    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991087    2559 flags.go:64] FLAG: --event-burst="100"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991090    2559 flags.go:64] FLAG: --event-qps="50"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991093    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991096    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991099    2559 flags.go:64] FLAG: --eviction-hard=""
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991108    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991111    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991114    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991117    2559 flags.go:64] FLAG: --eviction-soft=""
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991122    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991126    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 04 03:09:42.997580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991129    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991132    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991135    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991138    2559 flags.go:64] FLAG: --fail-swap-on="true"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991141    2559 flags.go:64] FLAG: --feature-gates=""
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991145    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991148    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991151    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991154    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991157    2559 flags.go:64] FLAG: --healthz-port="10248"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991161    2559 flags.go:64] FLAG: --help="false"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991164    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-134-110.ec2.internal"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991167    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991170    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991173    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991176    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991179    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991182    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991185    2559 flags.go:64] FLAG: --image-service-endpoint=""
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991188    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991191    2559 flags.go:64] FLAG: --kube-api-burst="100"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991195    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991198    2559 flags.go:64] FLAG: --kube-api-qps="50"
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991201    2559 flags.go:64] FLAG: --kube-reserved=""
Aug 04 03:09:42.998267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991204    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991206    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991210    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991212    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991215    2559 flags.go:64] FLAG: --lock-file=""
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991218    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991222    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991225    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991231    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991234    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991237    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991240    2559 flags.go:64] FLAG: --logging-format="text"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991243    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991255    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991259    2559 flags.go:64] FLAG: --manifest-url=""
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991261    2559 flags.go:64] FLAG: --manifest-url-header=""
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991266    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991271    2559 flags.go:64] FLAG: --max-open-files="1000000"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991275    2559 flags.go:64] FLAG: --max-pods="110"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991278    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991281    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991284    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991287    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991290    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991293    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 04 03:09:42.999046 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991296    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991303    2559 flags.go:64] FLAG: --node-status-max-images="50"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991307    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991310    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991313    2559 flags.go:64] FLAG: --pod-cidr=""
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991317    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991323    2559 flags.go:64] FLAG: --pod-manifest-path=""
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991326    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991329    2559 flags.go:64] FLAG: --pods-per-core="0"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991332    2559 flags.go:64] FLAG: --port="10250"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991335    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991338    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d115d9d3d728bfc5"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991341    2559 flags.go:64] FLAG: --qos-reserved=""
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991344    2559 flags.go:64] FLAG: --read-only-port="10255"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991349    2559 flags.go:64] FLAG: --register-node="true"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991352    2559 flags.go:64] FLAG: --register-with-taints=""
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991356    2559 flags.go:64] FLAG: --registry-burst="10"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991359    2559 flags.go:64] FLAG: --registry-qps="5"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991362    2559 flags.go:64] FLAG: --reserved-cpus=""
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991364    2559 flags.go:64] FLAG: --reserved-memory=""
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991368    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991371    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991374    2559 flags.go:64] FLAG: --rotate-certificates="false"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991377    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991382    2559 flags.go:64] FLAG: --runonce="false"
Aug 04 03:09:43.000018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991385    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991388    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991392    2559 flags.go:64] FLAG: --seccomp-default="false"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991394    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991397    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991400    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991404    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991407    2559 flags.go:64] FLAG: --storage-driver-password="root"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991409    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991412    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991415    2559 flags.go:64] FLAG: --storage-driver-user="root"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991418    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991421    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991425    2559 flags.go:64] FLAG: --system-cgroups=""
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991428    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991434    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991437    2559 flags.go:64] FLAG: --tls-cert-file=""
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991440    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991444    2559 flags.go:64] FLAG: --tls-min-version=""
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991447    2559 flags.go:64] FLAG: --tls-private-key-file=""
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991449    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991454    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991457    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991460    2559 flags.go:64] FLAG: --v="2"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991465    2559 flags.go:64] FLAG: --version="false"
Aug 04 03:09:43.000661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991469    2559 flags.go:64] FLAG: --vmodule=""
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991476    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.991479    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991631    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991636    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991639    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991643    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991647    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991649    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991653    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991655    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991658    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991661    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991663    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991666    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991669    2559 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991672    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991674    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991677    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991679    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 03:09:43.001276 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991682    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991685    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991688    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991691    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991693    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991696    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991699    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991701    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991705    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991707    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991710    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991712    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991715    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991718    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991720    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991723    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991725    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991728    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991732    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991735    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 03:09:43.001783 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991738    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991740    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991743    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991745    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991748    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991750    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991753    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991755    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991758    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991761    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991763    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991766    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991768    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991771    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991774    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991776    2559 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991779    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991782    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991784    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991787    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 03:09:43.002304 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991790    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991793    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991795    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991798    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991800    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991803    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991805    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991808    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991811    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991813    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991817    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991820    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991822    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991825    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991827    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991832    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991835    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991838    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991841    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991845    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 03:09:43.002807 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991848    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991851    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991854    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991856    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991860    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991863    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991867    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991871    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991874    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991876    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991879    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991881    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991885    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991888    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991890    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991893    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991895    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991898    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991900    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 03:09:43.003313 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991903    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991906    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:42.991909    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:42.992455    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.000751    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.000771    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.000793    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.000803    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000866    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000871    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000875    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000879    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000882    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000885    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000889    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 03:09:43.003782 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000891    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000894    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000897    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000900    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000902    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000905    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000908    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000910    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000913    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000915    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000918    2559 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000921    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000923    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000925    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000928    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000931    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000933    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000936    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000939    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000947    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 03:09:43.004207 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000950    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000954    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000960    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000963    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000967    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000970    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000974    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000978    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000981    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000984    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000987    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000990    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000992    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000995    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.000998    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001000    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001003    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001006    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001008    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 03:09:43.004732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001010    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001013    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001016    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001019    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001021    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001024    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001027    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001030    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001033    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001037    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001040    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001043    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001046    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001048    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001051    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001054    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001057    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001060    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 03:09:43.005210 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001063    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001065    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001068    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001070    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001073    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001075    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001078    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001080    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001083    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001085    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001088    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001090    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001093    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001095    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001098    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001102    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001106    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001108    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001111    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001114    2559 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 03:09:43.005764 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001116    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001119    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001121    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001124    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001126    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001129    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001131    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001134    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001136    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001140    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001142    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001145    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001148    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001150    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001153    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 03:09:43.006273 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.001158    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001284    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001289    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001292    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001296    2559 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001298    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001301    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001304    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001307    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001310    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001314    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001318    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001321    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001324    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001327    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001330    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001332    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001335    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001340    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 03:09:43.006864 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001343    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001346    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001349    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001352    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001355    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001357    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001360    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001364    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001367    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001369    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001372    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001375    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001377    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001380    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001382    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001384    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001387    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001389    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001392    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 03:09:43.007525 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001394    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001396    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001399    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001402    2559 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001404    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001407    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001409    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001411    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001414    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001417    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001420    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001422    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001425    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001428    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001430    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001433    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001435    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001438    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001440    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001443    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 03:09:43.008152 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001445    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001448    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001451    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001453    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001458    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001461    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001464    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001466    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001469    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001471    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001474    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001477    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001479    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001482    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001485    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001488    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001490    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001493    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001496    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 03:09:43.008953 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001498    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001501    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001504    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001506    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001509    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001512    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001515    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001517    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001520    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001522    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001525    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001527    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001530    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001533    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001536    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001555    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001558    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001561    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001564    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001566    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 03:09:43.009580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001569    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 03:09:43.010061 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001572    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 03:09:43.010061 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:43.001575    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 03:09:43.010061 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.001580    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 03:09:43.010061 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.001728    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 04 03:09:43.010061 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.003756    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 04 03:09:43.010061 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.005533    2559 server.go:1019] "Starting client certificate rotation"
Aug 04 03:09:43.010061 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.005662    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 03:09:43.010061 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.005717    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 03:09:43.010061 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.007959    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 03:09:43.010061 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.009239    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 03:09:43.023014 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.022991    2559 log.go:25] "Validated CRI v1 runtime API"
Aug 04 03:09:43.025166 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.025147    2559 log.go:25] "Validated CRI v1 image API"
Aug 04 03:09:43.026198 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.026184    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 04 03:09:43.028793 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.028774    2559 fs.go:135] Filesystem UUIDs: map[14ea04ef-61fa-4af2-a27d-09d6c2f0c4e1:/dev/nvme0n1p4 2b648d9a-70d3-48cc-8647-3de6e7044cc5:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Aug 04 03:09:43.028840 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.028793    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 04 03:09:43.032076 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.032057    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 03:09:43.034208 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.034075    2559 manager.go:216] Machine: {Timestamp:2026-08-04 03:09:43.032795786 +0000 UTC m=+0.330936286 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2499996 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f630a4e1436a66b87bafd6036c8bd SystemUUID:ec2f630a-4e14-36a6-6b87-bafd6036c8bd BootID:1609a782-c263-4cc1-9931-d05448633149 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:40:8d:2e:b8:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:40:8d:2e:b8:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:db:f9:b7:ca:74 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 04 03:09:43.034208 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.034200    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 04 03:09:43.034340 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.034291    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 04 03:09:43.037581 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.037554    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 04 03:09:43.037728 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.037584    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-110.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 04 03:09:43.038347 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.038337    2559 topology_manager.go:138] "Creating topology manager with none policy"
Aug 04 03:09:43.038377 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.038349    2559 container_manager_linux.go:309] "Creating device plugin manager"
Aug 04 03:09:43.038377 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.038357    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 03:09:43.039106 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.039096    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 03:09:43.040203 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.040192    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 04 03:09:43.043823 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.043809    2559 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 03:09:43.043958 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.043949    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 04 03:09:43.043994 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.043979    2559 kubelet.go:486] "Attempting to sync node with API server"
Aug 04 03:09:43.043994 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.043989    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 04 03:09:43.044050 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.044003    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 04 03:09:43.044050 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.044014    2559 kubelet.go:398] "Adding apiserver pod source"
Aug 04 03:09:43.044050 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.044024    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 04 03:09:43.044497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.044480    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 03:09:43.044561 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.044505    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 03:09:43.047893 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.047871    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 04 03:09:43.049229 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.049215    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 04 03:09:43.049229 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.049245    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 04 03:09:43.049860 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.049848    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 04 03:09:43.049932 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.049866    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 04 03:09:43.049932 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.049872    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 04 03:09:43.049932 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.049878    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 04 03:09:43.049932 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.049886    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 04 03:09:43.049932 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.049892    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 04 03:09:43.050068 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.050058    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 04 03:09:43.050068 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.050064    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 04 03:09:43.050121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.050071    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 04 03:09:43.050121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.050077    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 04 03:09:43.050121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.050086    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 04 03:09:43.050121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.050093    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 04 03:09:43.051650 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.051639    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 04 03:09:43.051650 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.051649    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 04 03:09:43.054316 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.054292    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4hrmm"
Aug 04 03:09:43.055088 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.055075    2559 server.go:1268] "Started kubelet"
Aug 04 03:09:43.055188 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.055162    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 04 03:09:43.055226 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.055155    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 04 03:09:43.055299 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.055278    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 04 03:09:43.055965 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.055944    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-110.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 04 03:09:43.056056 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.055991    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 04 03:09:43.056071 ip-10-0-134-110 systemd[1]: Started Kubernetes Kubelet.
Aug 04 03:09:43.056222 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.056182    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 04 03:09:43.056492 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.056449    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 04 03:09:43.057507 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.057496    2559 server.go:310] "Adding debug handlers to kubelet server"
Aug 04 03:09:43.060131 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.060105    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 04 03:09:43.061018 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.059877    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-110.ec2.internal.18c87b724a46dead  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-110.ec2.internal,UID:ip-10-0-134-110.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-110.ec2.internal,},FirstTimestamp:2026-08-04 03:09:43.055048365 +0000 UTC m=+0.353188844,LastTimestamp:2026-08-04 03:09:43.055048365 +0000 UTC m=+0.353188844,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-110.ec2.internal,}"
Aug 04 03:09:43.062731 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.062713    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 04 03:09:43.062731 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.062725    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 04 03:09:43.062858 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.062810    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:43.063414 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063397    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 04 03:09:43.063414 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063416    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 04 03:09:43.063576 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063389    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 04 03:09:43.063576 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063498    2559 reconstruct.go:97] "Volume reconstruction finished"
Aug 04 03:09:43.063576 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063505    2559 reconciler.go:29] "Reconciler: start to sync state"
Aug 04 03:09:43.063700 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063601    2559 factory.go:55] Registering systemd factory
Aug 04 03:09:43.063700 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063616    2559 factory.go:223] Registration of the systemd container factory successfully
Aug 04 03:09:43.063849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063832    2559 factory.go:153] Registering CRI-O factory
Aug 04 03:09:43.063849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063850    2559 factory.go:223] Registration of the crio container factory successfully
Aug 04 03:09:43.063992 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063900    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 04 03:09:43.063992 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063924    2559 factory.go:103] Registering Raw factory
Aug 04 03:09:43.063992 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.063938    2559 manager.go:1196] Started watching for new ooms in manager
Aug 04 03:09:43.064334 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.064320    2559 manager.go:349] Starting recovery of all containers
Aug 04 03:09:43.065502 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.065479    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4hrmm"
Aug 04 03:09:43.068100 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.068057    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 04 03:09:43.068249 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.068183    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 04 03:09:43.078244 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.078223    2559 manager.go:354] Recovery completed
Aug 04 03:09:43.079599 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.079496    2559 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Aug 04 03:09:43.082590 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.082577    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 03:09:43.084424 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.084373    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 03:09:43.084424 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.084407    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 03:09:43.084424 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.084423    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientPID"
Aug 04 03:09:43.085070 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.085056    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 04 03:09:43.085070 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.085069    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 04 03:09:43.085180 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.085084    2559 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 03:09:43.085180 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.085126    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 03:09:43.085849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.085835    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 03:09:43.085903 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.085863    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 03:09:43.085903 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.085878    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientPID"
Aug 04 03:09:43.086696 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.086684    2559 policy_none.go:49] "None policy: Start"
Aug 04 03:09:43.086747 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.086701    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 04 03:09:43.086747 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.086711    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 04 03:09:43.087951 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.087940    2559 policy_none.go:47] "Start"
Aug 04 03:09:43.140432 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.140253    2559 manager.go:337] "Starting Device Plugin manager"
Aug 04 03:09:43.140574 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.140455    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 04 03:09:43.140574 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.140470    2559 server.go:87] "Starting device plugin registration server"
Aug 04 03:09:43.140762 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.140751    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 04 03:09:43.140794 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.140766    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 04 03:09:43.140961 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.140946    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 04 03:09:43.141258 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.141019    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 04 03:09:43.141258 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.141029    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 04 03:09:43.141524 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.141505    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 04 03:09:43.141610 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.141570    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:43.230211 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.230170    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 04 03:09:43.231429 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.231409    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 04 03:09:43.231525 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.231449    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 04 03:09:43.231525 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.231476    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 04 03:09:43.231525 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.231511    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 04 03:09:43.233920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.233900    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 03:09:43.240857 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.240825    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 03:09:43.241785 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.241768    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 03:09:43.241883 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.241804    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 03:09:43.241883 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.241814    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientPID"
Aug 04 03:09:43.241883 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.241838    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.250934 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.250913    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.251047 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.250942    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-110.ec2.internal\": node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:43.266978 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.266950    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:43.332423 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.332389    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-110.ec2.internal"]
Aug 04 03:09:43.332567 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.332482    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 03:09:43.333953 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.333934    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 03:09:43.334065 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.333962    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 03:09:43.334065 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.333972    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientPID"
Aug 04 03:09:43.335458 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.335411    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 03:09:43.335631 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.335616    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.335690 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.335652    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 03:09:43.336215 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.336199    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 03:09:43.336304 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.336226    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 03:09:43.336304 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.336240    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientPID"
Aug 04 03:09:43.336634 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.336616    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 03:09:43.336686 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.336646    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 03:09:43.336686 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.336657    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientPID"
Aug 04 03:09:43.337445 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.337430    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.337528 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.337460    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 03:09:43.338404 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.338383    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 03:09:43.338483 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.338422    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 03:09:43.338483 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.338437    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeHasSufficientPID"
Aug 04 03:09:43.358379 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.358356    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-110.ec2.internal\" not found" node="ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.363176 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.363157    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-110.ec2.internal\" not found" node="ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.364661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.364640    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/314c6e474f45a46df975dbb8866f1f60-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal\" (UID: \"314c6e474f45a46df975dbb8866f1f60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.364752 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.364667    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/314c6e474f45a46df975dbb8866f1f60-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal\" (UID: \"314c6e474f45a46df975dbb8866f1f60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.364752 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.364693    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ce6294aa3b6a7973b0262394af529b36-config\") pod \"kube-apiserver-proxy-ip-10-0-134-110.ec2.internal\" (UID: \"ce6294aa3b6a7973b0262394af529b36\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.367730 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.367713    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:43.465673 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.465638    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/314c6e474f45a46df975dbb8866f1f60-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal\" (UID: \"314c6e474f45a46df975dbb8866f1f60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.465673 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.465677    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/314c6e474f45a46df975dbb8866f1f60-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal\" (UID: \"314c6e474f45a46df975dbb8866f1f60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.465898 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.465703    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ce6294aa3b6a7973b0262394af529b36-config\") pod \"kube-apiserver-proxy-ip-10-0-134-110.ec2.internal\" (UID: \"ce6294aa3b6a7973b0262394af529b36\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.465898 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.465732    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ce6294aa3b6a7973b0262394af529b36-config\") pod \"kube-apiserver-proxy-ip-10-0-134-110.ec2.internal\" (UID: \"ce6294aa3b6a7973b0262394af529b36\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.465898 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.465735    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/314c6e474f45a46df975dbb8866f1f60-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal\" (UID: \"314c6e474f45a46df975dbb8866f1f60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.465898 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.465730    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/314c6e474f45a46df975dbb8866f1f60-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal\" (UID: \"314c6e474f45a46df975dbb8866f1f60\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.468743 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.468721    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:43.569628 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.569583    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:43.660884 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.660812    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.665886 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:43.665871    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:43.669829 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.669802    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:43.770446 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.770403    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:43.870917 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.870879    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:43.971639 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:43.971569    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:44.004879 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:44.004861    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 04 03:09:44.005296 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:44.004993    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 03:09:44.044296 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:44.044268    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 03:09:44.071941 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:44.071916    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-110.ec2.internal\" not found"
Aug 04 03:09:44.071941 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:44.071908    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 03:04:43 +0000 UTC" deadline="2028-05-07 09:25:29.421176178 +0000 UTC"
Aug 04 03:09:44.071941 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:44.071945    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15414h15m45.349234923s"
Aug 04 03:09:44.094474 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:44.094449    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 03:09:44.163302 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:44.163270    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:44.176013 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:44.175991    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 03:09:44.177258 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:44.177243    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-110.ec2.internal"
Aug 04 03:09:44.184234 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:44.184213    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 03:09:44.281450 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:44.281413    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6294aa3b6a7973b0262394af529b36.slice/crio-8060fd9841929daeccd7191848dca28af045711df1adf4cfe4bad1df4e129f7d WatchSource:0}: Error finding container 8060fd9841929daeccd7191848dca28af045711df1adf4cfe4bad1df4e129f7d: Status 404 returned error can't find the container with id 8060fd9841929daeccd7191848dca28af045711df1adf4cfe4bad1df4e129f7d
Aug 04 03:09:44.285703 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:44.285681    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 03:09:44.308934 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:44.308903    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod314c6e474f45a46df975dbb8866f1f60.slice/crio-ce612ec36a942669030398db22cd3a7e7db5601477a3968888f5f3a93a9d13fb WatchSource:0}: Error finding container ce612ec36a942669030398db22cd3a7e7db5601477a3968888f5f3a93a9d13fb: Status 404 returned error can't find the container with id ce612ec36a942669030398db22cd3a7e7db5601477a3968888f5f3a93a9d13fb
Aug 04 03:09:45.008056 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.008026    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 03:09:45.045041 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.045010    2559 apiserver.go:52] "Watching apiserver"
Aug 04 03:09:45.051999 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.051969    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 04 03:09:45.053402 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.053368    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-dn58s","openshift-multus/network-metrics-daemon-xlcd8","openshift-network-diagnostics/network-check-target-6lt8h","openshift-network-operator/iptables-alerter-rgclr","kube-system/global-pull-secret-syncer-xbn5d","kube-system/kube-apiserver-proxy-ip-10-0-134-110.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw","openshift-image-registry/node-ca-9m9ks","openshift-multus/multus-dj667","openshift-ovn-kubernetes/ovnkube-node-kd7zs","kube-system/konnectivity-agent-2svj7","openshift-cluster-node-tuning-operator/tuned-6nqfs","openshift-dns/node-resolver-f4mrm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal"]
Aug 04 03:09:45.055047 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.055027    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.056141 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.056122    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f4mrm"
Aug 04 03:09:45.057376 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.057346    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:45.057465 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.057431    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:09:45.058721 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.058289    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 04 03:09:45.058721 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.058374    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k4jxk\""
Aug 04 03:09:45.058721 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.058630    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rgclr"
Aug 04 03:09:45.058721 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.058637    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5dskl\""
Aug 04 03:09:45.059914 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.059896    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:45.059995 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.059957    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:09:45.060337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.060162    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 04 03:09:45.060337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.060181    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 04 03:09:45.060465 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.060418    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 04 03:09:45.060465 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.060418    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 04 03:09:45.060945 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.060927    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9m9ks"
Aug 04 03:09:45.061364 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.061346    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 04 03:09:45.061652 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.061633    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 04 03:09:45.061915 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.061634    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 04 03:09:45.062520 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.062498    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zzx7d\""
Aug 04 03:09:45.062806 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.062778    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.062894 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.062834    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 04 03:09:45.063049 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.063032    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 04 03:09:45.063157 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.063138    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lcmcw\""
Aug 04 03:09:45.063506 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.063487    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 04 03:09:45.063685 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.063666    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 04 03:09:45.064924 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.064894    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.066224 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.066206    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 04 03:09:45.066440 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.066421    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 04 03:09:45.066536 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.066457    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 04 03:09:45.066645 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.066628    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 04 03:09:45.066702 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.066647    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:45.066751 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.066713    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qpfc8\""
Aug 04 03:09:45.066751 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.066720    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:09:45.067076 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.067059    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 04 03:09:45.067150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.067133    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-brxhp\""
Aug 04 03:09:45.067337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.067320    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 04 03:09:45.067641 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.067623    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.068855 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.068835    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:09:45.070112 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.070084    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 04 03:09:45.070325 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.070243    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.070325 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.070274    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 04 03:09:45.070591 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.070448    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 04 03:09:45.074909 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.071104    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 04 03:09:45.074909 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.071498    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 04 03:09:45.074909 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.071737    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 04 03:09:45.074909 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.071944    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rrlb7\""
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075382    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6864c31c-d3e5-4224-beda-e85b0d20868d-tmp-dir\") pod \"node-resolver-f4mrm\" (UID: \"6864c31c-d3e5-4224-beda-e85b0d20868d\") " pod="openshift-dns/node-resolver-f4mrm"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075425    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e1ea832d-816f-48c5-b6aa-1cefcae84ebe-iptables-alerter-script\") pod \"iptables-alerter-rgclr\" (UID: \"e1ea832d-816f-48c5-b6aa-1cefcae84ebe\") " pod="openshift-network-operator/iptables-alerter-rgclr"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075459    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1ea832d-816f-48c5-b6aa-1cefcae84ebe-host-slash\") pod \"iptables-alerter-rgclr\" (UID: \"e1ea832d-816f-48c5-b6aa-1cefcae84ebe\") " pod="openshift-network-operator/iptables-alerter-rgclr"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075491    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-run-k8s-cni-cncf-io\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075525    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-registration-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-device-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075596    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-multus-cni-dir\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075626    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-os-release\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075686    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s85v\" (UniqueName: \"kubernetes.io/projected/df20bbca-eb3a-44c3-be85-289103e0f04d-kube-api-access-2s85v\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075718    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075773    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-run-multus-certs\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075804    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a2ea3c8-88ca-4802-987b-be5942899884-serviceca\") pod \"node-ca-9m9ks\" (UID: \"7a2ea3c8-88ca-4802-987b-be5942899884\") " pod="openshift-image-registry/node-ca-9m9ks"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075838    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2hf7\" (UniqueName: \"kubernetes.io/projected/7a2ea3c8-88ca-4802-987b-be5942899884-kube-api-access-x2hf7\") pod \"node-ca-9m9ks\" (UID: \"7a2ea3c8-88ca-4802-987b-be5942899884\") " pod="openshift-image-registry/node-ca-9m9ks"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075876    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clxcw\" (UniqueName: \"kubernetes.io/projected/6864c31c-d3e5-4224-beda-e85b0d20868d-kube-api-access-clxcw\") pod \"node-resolver-f4mrm\" (UID: \"6864c31c-d3e5-4224-beda-e85b0d20868d\") " pod="openshift-dns/node-resolver-f4mrm"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075907    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7dcf6319-3c75-4937-8f57-4c2735996721-cni-binary-copy\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075933    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-hostroot\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.076195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075945    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 04 03:09:45.076889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075967    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-kubelet-config\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:45.076889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.076010    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s77xt\" (UniqueName: \"kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt\") pod \"network-check-target-6lt8h\" (UID: \"409a5d1e-64ba-40a1-9bfb-57211e03d6ba\") " pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:45.076889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.076039    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a2ea3c8-88ca-4802-987b-be5942899884-host\") pod \"node-ca-9m9ks\" (UID: \"7a2ea3c8-88ca-4802-987b-be5942899884\") " pod="openshift-image-registry/node-ca-9m9ks"
Aug 04 03:09:45.076889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.076068    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-system-cni-dir\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.076889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.076109    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/df20bbca-eb3a-44c3-be85-289103e0f04d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.076889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.076136    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-sys-fs\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.076889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.076171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f68j\" (UniqueName: \"kubernetes.io/projected/842cb690-dfeb-49ec-aef1-401993115383-kube-api-access-8f68j\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.076889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.076434    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 04 03:09:45.078569 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.077528    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bqgvj\""
Aug 04 03:09:45.078569 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.075878    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 04 03:09:45.078569 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.077668    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 04 03:09:45.078569 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.076175    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-87ldg\""
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.078851    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-cnibin\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.078894    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-run-netns\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.078915    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-os-release\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.078944    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-etc-selinux\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.078976    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6864c31c-d3e5-4224-beda-e85b0d20868d-hosts-file\") pod \"node-resolver-f4mrm\" (UID: \"6864c31c-d3e5-4224-beda-e85b0d20868d\") " pod="openshift-dns/node-resolver-f4mrm"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079009    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-multus-socket-dir-parent\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079041    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-var-lib-cni-bin\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079069    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-multus-conf-dir\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079098    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7dcf6319-3c75-4937-8f57-4c2735996721-multus-daemon-config\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df20bbca-eb3a-44c3-be85-289103e0f04d-cni-binary-copy\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/df20bbca-eb3a-44c3-be85-289103e0f04d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079187    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-socket-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079222    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4rlz\" (UniqueName: \"kubernetes.io/projected/e1ea832d-816f-48c5-b6aa-1cefcae84ebe-kube-api-access-j4rlz\") pod \"iptables-alerter-rgclr\" (UID: \"e1ea832d-816f-48c5-b6aa-1cefcae84ebe\") " pod="openshift-network-operator/iptables-alerter-rgclr"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079259    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-var-lib-cni-multus\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079290    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-var-lib-kubelet\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.079790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079324    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-system-cni-dir\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.080451 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-cnibin\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.080451 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079386    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.080451 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079455    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:45.080451 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079502    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-etc-kubernetes\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.080451 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079555    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvmvz\" (UniqueName: \"kubernetes.io/projected/7dcf6319-3c75-4937-8f57-4c2735996721-kube-api-access-kvmvz\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.080451 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.079621    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-dbus\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:45.081812 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.081792    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 03:09:45.102868 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.102839    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ptmgh"
Aug 04 03:09:45.108739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.108713    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ptmgh"
Aug 04 03:09:45.164468 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.164442    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 04 03:09:45.180015 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.179971    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7dcf6319-3c75-4937-8f57-4c2735996721-multus-daemon-config\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.180179 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180019    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df20bbca-eb3a-44c3-be85-289103e0f04d-cni-binary-copy\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.180179 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180049    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/df20bbca-eb3a-44c3-be85-289103e0f04d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.180179 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180078    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-sysctl-d\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.180179 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4rlz\" (UniqueName: \"kubernetes.io/projected/e1ea832d-816f-48c5-b6aa-1cefcae84ebe-kube-api-access-j4rlz\") pod \"iptables-alerter-rgclr\" (UID: \"e1ea832d-816f-48c5-b6aa-1cefcae84ebe\") " pod="openshift-network-operator/iptables-alerter-rgclr"
Aug 04 03:09:45.180179 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180144    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.180179 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180169    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:45.180395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180194    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e99772e-31e2-4eda-88af-0fc77fb0156e-env-overrides\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.180395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180218    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbc9759d-10ad-4442-9a87-868e5ba4749a-tmp\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.180395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180242    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6864c31c-d3e5-4224-beda-e85b0d20868d-tmp-dir\") pod \"node-resolver-f4mrm\" (UID: \"6864c31c-d3e5-4224-beda-e85b0d20868d\") " pod="openshift-dns/node-resolver-f4mrm"
Aug 04 03:09:45.180395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180266    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1ea832d-816f-48c5-b6aa-1cefcae84ebe-host-slash\") pod \"iptables-alerter-rgclr\" (UID: \"e1ea832d-816f-48c5-b6aa-1cefcae84ebe\") " pod="openshift-network-operator/iptables-alerter-rgclr"
Aug 04 03:09:45.180395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180291    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-run-k8s-cni-cncf-io\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.180395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180317    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-device-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.180395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180341    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-run-netns\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.180395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180387    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72lbh\" (UniqueName: \"kubernetes.io/projected/57750f19-0d04-432e-94f1-d0d0fde883bf-kube-api-access-72lbh\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:45.180708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfxlb\" (UniqueName: \"kubernetes.io/projected/bbc9759d-10ad-4442-9a87-868e5ba4749a-kube-api-access-xfxlb\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.180708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-cni-netd\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.180708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-sysconfig\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.180708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180486    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-host\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.180708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180510    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a2ea3c8-88ca-4802-987b-be5942899884-serviceca\") pod \"node-ca-9m9ks\" (UID: \"7a2ea3c8-88ca-4802-987b-be5942899884\") " pod="openshift-image-registry/node-ca-9m9ks"
Aug 04 03:09:45.180708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2hf7\" (UniqueName: \"kubernetes.io/projected/7a2ea3c8-88ca-4802-987b-be5942899884-kube-api-access-x2hf7\") pod \"node-ca-9m9ks\" (UID: \"7a2ea3c8-88ca-4802-987b-be5942899884\") " pod="openshift-image-registry/node-ca-9m9ks"
Aug 04 03:09:45.180708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180583    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:45.180708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180607    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-system-cni-dir\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.180708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180634    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/df20bbca-eb3a-44c3-be85-289103e0f04d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.180708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180660    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-log-socket\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.180708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180660    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7dcf6319-3c75-4937-8f57-4c2735996721-multus-daemon-config\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.181106 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180712    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-device-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.181106 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.180744    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.181309 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181290    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/df20bbca-eb3a-44c3-be85-289103e0f04d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.181342 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181306    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df20bbca-eb3a-44c3-be85-289103e0f04d-cni-binary-copy\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181665    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-system-cni-dir\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.181690    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181702    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e99772e-31e2-4eda-88af-0fc77fb0156e-ovnkube-config\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181714    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-run-k8s-cni-cncf-io\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181672    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181728    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e99772e-31e2-4eda-88af-0fc77fb0156e-ovn-node-metrics-cert\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.181758    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret podName:550248ce-69c9-4f77-9e9e-f6f0a509f0c4 nodeName:}" failed. No retries permitted until 2026-08-04 03:09:45.681738183 +0000 UTC m=+2.979878661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret") pod "global-pull-secret-syncer-xbn5d" (UID: "550248ce-69c9-4f77-9e9e-f6f0a509f0c4") : object "kube-system"/"original-pull-secret" not registered
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181762    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1ea832d-816f-48c5-b6aa-1cefcae84ebe-host-slash\") pod \"iptables-alerter-rgclr\" (UID: \"e1ea832d-816f-48c5-b6aa-1cefcae84ebe\") " pod="openshift-network-operator/iptables-alerter-rgclr"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7a2ea3c8-88ca-4802-987b-be5942899884-serviceca\") pod \"node-ca-9m9ks\" (UID: \"7a2ea3c8-88ca-4802-987b-be5942899884\") " pod="openshift-image-registry/node-ca-9m9ks"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181799    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48fcbb9a-7f25-411d-834f-00594bc5fb4f-agent-certs\") pod \"konnectivity-agent-2svj7\" (UID: \"48fcbb9a-7f25-411d-834f-00594bc5fb4f\") " pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181827    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-lib-modules\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181860    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-os-release\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181885    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-etc-selinux\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181902    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6864c31c-d3e5-4224-beda-e85b0d20868d-tmp-dir\") pod \"node-resolver-f4mrm\" (UID: \"6864c31c-d3e5-4224-beda-e85b0d20868d\") " pod="openshift-dns/node-resolver-f4mrm"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48fcbb9a-7f25-411d-834f-00594bc5fb4f-konnectivity-ca\") pod \"konnectivity-agent-2svj7\" (UID: \"48fcbb9a-7f25-411d-834f-00594bc5fb4f\") " pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181933    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-var-lib-kubelet\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.181980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181946    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-os-release\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181959    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6864c31c-d3e5-4224-beda-e85b0d20868d-hosts-file\") pod \"node-resolver-f4mrm\" (UID: \"6864c31c-d3e5-4224-beda-e85b0d20868d\") " pod="openshift-dns/node-resolver-f4mrm"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.181982    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-multus-socket-dir-parent\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182004    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-socket-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182010    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-etc-selinux\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182026    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e99772e-31e2-4eda-88af-0fc77fb0156e-ovnkube-script-lib\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182058    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-var-lib-cni-multus\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182068    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6864c31c-d3e5-4224-beda-e85b0d20868d-hosts-file\") pod \"node-resolver-f4mrm\" (UID: \"6864c31c-d3e5-4224-beda-e85b0d20868d\") " pod="openshift-dns/node-resolver-f4mrm"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182094    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-multus-socket-dir-parent\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182105    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-var-lib-kubelet\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182119    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-var-lib-cni-multus\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182126    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/df20bbca-eb3a-44c3-be85-289103e0f04d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182135    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-system-cni-dir\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182154    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-var-lib-kubelet\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182161    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-cnibin\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182167    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-socket-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182187    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-systemd-units\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.182870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182195    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-system-cni-dir\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182235    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-run-systemd\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182240    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df20bbca-eb3a-44c3-be85-289103e0f04d-cnibin\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182263    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-etc-kubernetes\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182285    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvmvz\" (UniqueName: \"kubernetes.io/projected/7dcf6319-3c75-4937-8f57-4c2735996721-kube-api-access-kvmvz\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182311    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-dbus\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182336    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-etc-kubernetes\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182427    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-dbus\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182429    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-node-log\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182457    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-kubernetes\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182494    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-sys\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182556    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e1ea832d-816f-48c5-b6aa-1cefcae84ebe-iptables-alerter-script\") pod \"iptables-alerter-rgclr\" (UID: \"e1ea832d-816f-48c5-b6aa-1cefcae84ebe\") " pod="openshift-network-operator/iptables-alerter-rgclr"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182589    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-registration-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182637    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-registration-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182693    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-slash\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rss9m\" (UniqueName: \"kubernetes.io/projected/1e99772e-31e2-4eda-88af-0fc77fb0156e-kube-api-access-rss9m\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182808    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-modprobe-d\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.183628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182830    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-sysctl-conf\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182854    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-multus-cni-dir\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182881    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-os-release\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182907    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s85v\" (UniqueName: \"kubernetes.io/projected/df20bbca-eb3a-44c3-be85-289103e0f04d-kube-api-access-2s85v\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182931    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-kubelet\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.182981    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-run-openvswitch\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183003    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-run\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183051    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-tuned\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183053    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e1ea832d-816f-48c5-b6aa-1cefcae84ebe-iptables-alerter-script\") pod \"iptables-alerter-rgclr\" (UID: \"e1ea832d-816f-48c5-b6aa-1cefcae84ebe\") " pod="openshift-network-operator/iptables-alerter-rgclr"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183067    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-multus-cni-dir\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183074    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-run-multus-certs\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183097    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clxcw\" (UniqueName: \"kubernetes.io/projected/6864c31c-d3e5-4224-beda-e85b0d20868d-kube-api-access-clxcw\") pod \"node-resolver-f4mrm\" (UID: \"6864c31c-d3e5-4224-beda-e85b0d20868d\") " pod="openshift-dns/node-resolver-f4mrm"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183124    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7dcf6319-3c75-4937-8f57-4c2735996721-cni-binary-copy\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183172    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-hostroot\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183194    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-kubelet-config\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:45.184526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183213    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-os-release\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183218    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-run-multus-certs\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183296    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-run-ovn-kubernetes\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183328    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77xt\" (UniqueName: \"kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt\") pod \"network-check-target-6lt8h\" (UID: \"409a5d1e-64ba-40a1-9bfb-57211e03d6ba\") " pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183366    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a2ea3c8-88ca-4802-987b-be5942899884-host\") pod \"node-ca-9m9ks\" (UID: \"7a2ea3c8-88ca-4802-987b-be5942899884\") " pod="openshift-image-registry/node-ca-9m9ks"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-sys-fs\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183415    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f68j\" (UniqueName: \"kubernetes.io/projected/842cb690-dfeb-49ec-aef1-401993115383-kube-api-access-8f68j\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183440    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-run-ovn\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183481    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-cnibin\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183506    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-run-netns\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183531    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-var-lib-openvswitch\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183571    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-etc-openvswitch\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183634    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-cni-bin\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183688    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-systemd\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-var-lib-cni-bin\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183718    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-kubelet-config\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183744    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-multus-conf-dir\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.185314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183766    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-cnibin\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.186023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183799    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7a2ea3c8-88ca-4802-987b-be5942899884-host\") pod \"node-ca-9m9ks\" (UID: \"7a2ea3c8-88ca-4802-987b-be5942899884\") " pod="openshift-image-registry/node-ca-9m9ks"
Aug 04 03:09:45.186023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183838    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/842cb690-dfeb-49ec-aef1-401993115383-sys-fs\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.186023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183841    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7dcf6319-3c75-4937-8f57-4c2735996721-cni-binary-copy\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.186023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183838    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-run-netns\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.186023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183873    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-multus-conf-dir\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.186023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183901    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-hostroot\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.186023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.183912    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7dcf6319-3c75-4937-8f57-4c2735996721-host-var-lib-cni-bin\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.189796 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.189771    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 04 03:09:45.193450 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.193427    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2hf7\" (UniqueName: \"kubernetes.io/projected/7a2ea3c8-88ca-4802-987b-be5942899884-kube-api-access-x2hf7\") pod \"node-ca-9m9ks\" (UID: \"7a2ea3c8-88ca-4802-987b-be5942899884\") " pod="openshift-image-registry/node-ca-9m9ks"
Aug 04 03:09:45.193450 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.193441    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4rlz\" (UniqueName: \"kubernetes.io/projected/e1ea832d-816f-48c5-b6aa-1cefcae84ebe-kube-api-access-j4rlz\") pod \"iptables-alerter-rgclr\" (UID: \"e1ea832d-816f-48c5-b6aa-1cefcae84ebe\") " pod="openshift-network-operator/iptables-alerter-rgclr"
Aug 04 03:09:45.194635 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.194586    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 03:09:45.194635 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.194610    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 03:09:45.194635 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.194631    2559 projected.go:196] Error preparing data for projected volume kube-api-access-s77xt for pod openshift-network-diagnostics/network-check-target-6lt8h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:09:45.194840 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.194702    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt podName:409a5d1e-64ba-40a1-9bfb-57211e03d6ba nodeName:}" failed. No retries permitted until 2026-08-04 03:09:45.694681669 +0000 UTC m=+2.992822137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-s77xt" (UniqueName: "kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt") pod "network-check-target-6lt8h" (UID: "409a5d1e-64ba-40a1-9bfb-57211e03d6ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:09:45.196570 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.196495    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s85v\" (UniqueName: \"kubernetes.io/projected/df20bbca-eb3a-44c3-be85-289103e0f04d-kube-api-access-2s85v\") pod \"multus-additional-cni-plugins-dn58s\" (UID: \"df20bbca-eb3a-44c3-be85-289103e0f04d\") " pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.196920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.196898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvmvz\" (UniqueName: \"kubernetes.io/projected/7dcf6319-3c75-4937-8f57-4c2735996721-kube-api-access-kvmvz\") pod \"multus-dj667\" (UID: \"7dcf6319-3c75-4937-8f57-4c2735996721\") " pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.197171 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.197153    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f68j\" (UniqueName: \"kubernetes.io/projected/842cb690-dfeb-49ec-aef1-401993115383-kube-api-access-8f68j\") pod \"aws-ebs-csi-driver-node-8jsrw\" (UID: \"842cb690-dfeb-49ec-aef1-401993115383\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.198104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.198083    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clxcw\" (UniqueName: \"kubernetes.io/projected/6864c31c-d3e5-4224-beda-e85b0d20868d-kube-api-access-clxcw\") pod \"node-resolver-f4mrm\" (UID: \"6864c31c-d3e5-4224-beda-e85b0d20868d\") " pod="openshift-dns/node-resolver-f4mrm"
Aug 04 03:09:45.236798 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.236744    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal" event={"ID":"314c6e474f45a46df975dbb8866f1f60","Type":"ContainerStarted","Data":"ce612ec36a942669030398db22cd3a7e7db5601477a3968888f5f3a93a9d13fb"}
Aug 04 03:09:45.238474 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.238442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-110.ec2.internal" event={"ID":"ce6294aa3b6a7973b0262394af529b36","Type":"ContainerStarted","Data":"8060fd9841929daeccd7191848dca28af045711df1adf4cfe4bad1df4e129f7d"}
Aug 04 03:09:45.284867 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.284777    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e99772e-31e2-4eda-88af-0fc77fb0156e-ovnkube-script-lib\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.284867 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.284824    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-systemd-units\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.284867 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.284850    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-run-systemd\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.284873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-node-log\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.284897    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-kubernetes\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.284909    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-systemd-units\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.284919    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-sys\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.284964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-slash\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.284972    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-run-systemd\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.284976    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-node-log\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.284998    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-slash\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285014    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rss9m\" (UniqueName: \"kubernetes.io/projected/1e99772e-31e2-4eda-88af-0fc77fb0156e-kube-api-access-rss9m\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285028    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-kubernetes\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285037    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-sys\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285074    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-modprobe-d\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285098    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-sysctl-conf\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285126    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-kubelet\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285164    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-kubelet\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285243    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-run-openvswitch\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285254    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-modprobe-d\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285251    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-sysctl-conf\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-run-openvswitch\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285279    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-run\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285311    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-tuned\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285343    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-run-ovn-kubernetes\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285355    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-run\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285392    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-run-ovn\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285400    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-run-ovn-kubernetes\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285431    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-run-ovn\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285431    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-var-lib-openvswitch\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285472    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-etc-openvswitch\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285495    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-cni-bin\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285517    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-systemd\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.285704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285533    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e99772e-31e2-4eda-88af-0fc77fb0156e-ovnkube-script-lib\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285556    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-etc-openvswitch\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285588    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-sysctl-d\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285589    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-systemd\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285588    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-cni-bin\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285630    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-var-lib-openvswitch\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285634    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e99772e-31e2-4eda-88af-0fc77fb0156e-env-overrides\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285752    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbc9759d-10ad-4442-9a87-868e5ba4749a-tmp\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285698    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-sysctl-d\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285781    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-run-netns\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285817    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-run-netns\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72lbh\" (UniqueName: \"kubernetes.io/projected/57750f19-0d04-432e-94f1-d0d0fde883bf-kube-api-access-72lbh\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285872    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfxlb\" (UniqueName: \"kubernetes.io/projected/bbc9759d-10ad-4442-9a87-868e5ba4749a-kube-api-access-xfxlb\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285897    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-cni-netd\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285922    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-sysconfig\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285945    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-host\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285972    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286000    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-log-socket\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.286509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286020    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-sysconfig\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286025    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286026    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-host\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.285974    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e99772e-31e2-4eda-88af-0fc77fb0156e-env-overrides\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286051    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e99772e-31e2-4eda-88af-0fc77fb0156e-ovnkube-config\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286077    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e99772e-31e2-4eda-88af-0fc77fb0156e-ovn-node-metrics-cert\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286051    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-cni-netd\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286105    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-log-socket\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286110    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48fcbb9a-7f25-411d-834f-00594bc5fb4f-agent-certs\") pod \"konnectivity-agent-2svj7\" (UID: \"48fcbb9a-7f25-411d-834f-00594bc5fb4f\") " pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286085    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e99772e-31e2-4eda-88af-0fc77fb0156e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.286119    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286132    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-lib-modules\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286183    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48fcbb9a-7f25-411d-834f-00594bc5fb4f-konnectivity-ca\") pod \"konnectivity-agent-2svj7\" (UID: \"48fcbb9a-7f25-411d-834f-00594bc5fb4f\") " pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.286216    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs podName:57750f19-0d04-432e-94f1-d0d0fde883bf nodeName:}" failed. No retries permitted until 2026-08-04 03:09:45.786198717 +0000 UTC m=+3.084339189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs") pod "network-metrics-daemon-xlcd8" (UID: "57750f19-0d04-432e-94f1-d0d0fde883bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286234    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-var-lib-kubelet\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286306    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-var-lib-kubelet\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286413    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbc9759d-10ad-4442-9a87-868e5ba4749a-lib-modules\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.287337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286607    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e99772e-31e2-4eda-88af-0fc77fb0156e-ovnkube-config\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.288051 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.286747    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48fcbb9a-7f25-411d-834f-00594bc5fb4f-konnectivity-ca\") pod \"konnectivity-agent-2svj7\" (UID: \"48fcbb9a-7f25-411d-834f-00594bc5fb4f\") " pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:09:45.288051 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.287955    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbc9759d-10ad-4442-9a87-868e5ba4749a-etc-tuned\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.288175 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.288156    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbc9759d-10ad-4442-9a87-868e5ba4749a-tmp\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.288889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.288866    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e99772e-31e2-4eda-88af-0fc77fb0156e-ovn-node-metrics-cert\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.289470 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.289451    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48fcbb9a-7f25-411d-834f-00594bc5fb4f-agent-certs\") pod \"konnectivity-agent-2svj7\" (UID: \"48fcbb9a-7f25-411d-834f-00594bc5fb4f\") " pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:09:45.293296 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.293166    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rss9m\" (UniqueName: \"kubernetes.io/projected/1e99772e-31e2-4eda-88af-0fc77fb0156e-kube-api-access-rss9m\") pod \"ovnkube-node-kd7zs\" (UID: \"1e99772e-31e2-4eda-88af-0fc77fb0156e\") " pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.293469 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.293447    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfxlb\" (UniqueName: \"kubernetes.io/projected/bbc9759d-10ad-4442-9a87-868e5ba4749a-kube-api-access-xfxlb\") pod \"tuned-6nqfs\" (UID: \"bbc9759d-10ad-4442-9a87-868e5ba4749a\") " pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.295948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.295927    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72lbh\" (UniqueName: \"kubernetes.io/projected/57750f19-0d04-432e-94f1-d0d0fde883bf-kube-api-access-72lbh\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:45.316529 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.316500    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 03:09:45.369514 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.369485    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw"
Aug 04 03:09:45.381073 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.381046    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f4mrm"
Aug 04 03:09:45.389914 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.389889    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:09:45.398995 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.398967    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rgclr"
Aug 04 03:09:45.407620 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.407601    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9m9ks"
Aug 04 03:09:45.415267 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.415247    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dj667"
Aug 04 03:09:45.422874 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.422847    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dn58s"
Aug 04 03:09:45.429596 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.429569    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:09:45.434111 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.434091    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6nqfs"
Aug 04 03:09:45.689184 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.689095    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:45.689350 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.689261    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 03:09:45.689350 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.689338    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret podName:550248ce-69c9-4f77-9e9e-f6f0a509f0c4 nodeName:}" failed. No retries permitted until 2026-08-04 03:09:46.689317307 +0000 UTC m=+3.987457789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret") pod "global-pull-secret-syncer-xbn5d" (UID: "550248ce-69c9-4f77-9e9e-f6f0a509f0c4") : object "kube-system"/"original-pull-secret" not registered
Aug 04 03:09:45.790154 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.790119    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:45.790326 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:45.790168    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77xt\" (UniqueName: \"kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt\") pod \"network-check-target-6lt8h\" (UID: \"409a5d1e-64ba-40a1-9bfb-57211e03d6ba\") " pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:45.790326 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.790281    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:09:45.790326 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.790293    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 03:09:45.790326 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.790317    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 03:09:45.790454 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.790330    2559 projected.go:196] Error preparing data for projected volume kube-api-access-s77xt for pod openshift-network-diagnostics/network-check-target-6lt8h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:09:45.790454 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.790361    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs podName:57750f19-0d04-432e-94f1-d0d0fde883bf nodeName:}" failed. No retries permitted until 2026-08-04 03:09:46.790342429 +0000 UTC m=+4.088482896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs") pod "network-metrics-daemon-xlcd8" (UID: "57750f19-0d04-432e-94f1-d0d0fde883bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:09:45.790454 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:45.790385    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt podName:409a5d1e-64ba-40a1-9bfb-57211e03d6ba nodeName:}" failed. No retries permitted until 2026-08-04 03:09:46.790368289 +0000 UTC m=+4.088508767 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-s77xt" (UniqueName: "kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt") pod "network-check-target-6lt8h" (UID: "409a5d1e-64ba-40a1-9bfb-57211e03d6ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:09:46.111844 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.111810    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 03:04:45 +0000 UTC" deadline="2028-02-04 10:12:18.679453805 +0000 UTC"
Aug 04 03:09:46.112249 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.111848    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13183h2m32.56761131s"
Aug 04 03:09:46.118363 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:46.118337    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2ea3c8_88ca_4802_987b_be5942899884.slice/crio-5bbf373d539eb47fb89a9b82ee927d797912b5ee46393db96e780e0a29ccbbf3 WatchSource:0}: Error finding container 5bbf373d539eb47fb89a9b82ee927d797912b5ee46393db96e780e0a29ccbbf3: Status 404 returned error can't find the container with id 5bbf373d539eb47fb89a9b82ee927d797912b5ee46393db96e780e0a29ccbbf3
Aug 04 03:09:46.120401 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:46.120278    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf20bbca_eb3a_44c3_be85_289103e0f04d.slice/crio-b8d63d99085578a9ccaf2f6a3301239b8db58f7d29a4080c341945efb0ec13bb WatchSource:0}: Error finding container b8d63d99085578a9ccaf2f6a3301239b8db58f7d29a4080c341945efb0ec13bb: Status 404 returned error can't find the container with id b8d63d99085578a9ccaf2f6a3301239b8db58f7d29a4080c341945efb0ec13bb
Aug 04 03:09:46.123638 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:46.123424    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc9759d_10ad_4442_9a87_868e5ba4749a.slice/crio-9608acd921cece812943511d19f7a8d95f08969eb12b52c9a3f76dac57fae713 WatchSource:0}: Error finding container 9608acd921cece812943511d19f7a8d95f08969eb12b52c9a3f76dac57fae713: Status 404 returned error can't find the container with id 9608acd921cece812943511d19f7a8d95f08969eb12b52c9a3f76dac57fae713
Aug 04 03:09:46.124153 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:46.124128    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ea832d_816f_48c5_b6aa_1cefcae84ebe.slice/crio-c3562cd728f813ef01e1725d65f06817ca8c822f957cd75c1b96f83b8177db44 WatchSource:0}: Error finding container c3562cd728f813ef01e1725d65f06817ca8c822f957cd75c1b96f83b8177db44: Status 404 returned error can't find the container with id c3562cd728f813ef01e1725d65f06817ca8c822f957cd75c1b96f83b8177db44
Aug 04 03:09:46.124887 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:46.124863    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dcf6319_3c75_4937_8f57_4c2735996721.slice/crio-bfc3ce865670ba43deb1832a8bf37d2306c9f0260d9d17591942209f7f324f64 WatchSource:0}: Error finding container bfc3ce865670ba43deb1832a8bf37d2306c9f0260d9d17591942209f7f324f64: Status 404 returned error can't find the container with id bfc3ce865670ba43deb1832a8bf37d2306c9f0260d9d17591942209f7f324f64
Aug 04 03:09:46.125919 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:46.125897    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e99772e_31e2_4eda_88af_0fc77fb0156e.slice/crio-07aca7127ac2a648fae19ef909dde0b5c2fc99dfb9b0fa6fa8ae478aff51ff43 WatchSource:0}: Error finding container 07aca7127ac2a648fae19ef909dde0b5c2fc99dfb9b0fa6fa8ae478aff51ff43: Status 404 returned error can't find the container with id 07aca7127ac2a648fae19ef909dde0b5c2fc99dfb9b0fa6fa8ae478aff51ff43
Aug 04 03:09:46.127446 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:46.127424    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6864c31c_d3e5_4224_beda_e85b0d20868d.slice/crio-43bd6d96cb36a6f576deaf96c7c81110c0315655ca350745548b67a53b767014 WatchSource:0}: Error finding container 43bd6d96cb36a6f576deaf96c7c81110c0315655ca350745548b67a53b767014: Status 404 returned error can't find the container with id 43bd6d96cb36a6f576deaf96c7c81110c0315655ca350745548b67a53b767014
Aug 04 03:09:46.130036 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:09:46.130014    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod842cb690_dfeb_49ec_aef1_401993115383.slice/crio-a03a9b6ae03006ca41bec7b1a4827f576e16cb1922bb79499839f4f908f85d37 WatchSource:0}: Error finding container a03a9b6ae03006ca41bec7b1a4827f576e16cb1922bb79499839f4f908f85d37: Status 404 returned error can't find the container with id a03a9b6ae03006ca41bec7b1a4827f576e16cb1922bb79499839f4f908f85d37
Aug 04 03:09:46.232766 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.232734    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:46.232922 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:46.232873    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:09:46.242332 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.242297    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-110.ec2.internal" event={"ID":"ce6294aa3b6a7973b0262394af529b36","Type":"ContainerStarted","Data":"f064034db5dc60c15a3874c9070f67b3df372d7df5da39fe27593b7aaa44000f"}
Aug 04 03:09:46.243377 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.243349    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2svj7" event={"ID":"48fcbb9a-7f25-411d-834f-00594bc5fb4f","Type":"ContainerStarted","Data":"8a8b5c75c21d928b9c32b303473be42e77d0e89f97447ba9e873368288ccdd8c"}
Aug 04 03:09:46.244668 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.244648    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rgclr" event={"ID":"e1ea832d-816f-48c5-b6aa-1cefcae84ebe","Type":"ContainerStarted","Data":"c3562cd728f813ef01e1725d65f06817ca8c822f957cd75c1b96f83b8177db44"}
Aug 04 03:09:46.245768 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.245750    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6nqfs" event={"ID":"bbc9759d-10ad-4442-9a87-868e5ba4749a","Type":"ContainerStarted","Data":"9608acd921cece812943511d19f7a8d95f08969eb12b52c9a3f76dac57fae713"}
Aug 04 03:09:46.246875 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.246851    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dn58s" event={"ID":"df20bbca-eb3a-44c3-be85-289103e0f04d","Type":"ContainerStarted","Data":"b8d63d99085578a9ccaf2f6a3301239b8db58f7d29a4080c341945efb0ec13bb"}
Aug 04 03:09:46.247952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.247933    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9m9ks" event={"ID":"7a2ea3c8-88ca-4802-987b-be5942899884","Type":"ContainerStarted","Data":"5bbf373d539eb47fb89a9b82ee927d797912b5ee46393db96e780e0a29ccbbf3"}
Aug 04 03:09:46.249128 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.249107    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw" event={"ID":"842cb690-dfeb-49ec-aef1-401993115383","Type":"ContainerStarted","Data":"a03a9b6ae03006ca41bec7b1a4827f576e16cb1922bb79499839f4f908f85d37"}
Aug 04 03:09:46.250180 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.250160    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f4mrm" event={"ID":"6864c31c-d3e5-4224-beda-e85b0d20868d","Type":"ContainerStarted","Data":"43bd6d96cb36a6f576deaf96c7c81110c0315655ca350745548b67a53b767014"}
Aug 04 03:09:46.251314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.251295    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs" event={"ID":"1e99772e-31e2-4eda-88af-0fc77fb0156e","Type":"ContainerStarted","Data":"07aca7127ac2a648fae19ef909dde0b5c2fc99dfb9b0fa6fa8ae478aff51ff43"}
Aug 04 03:09:46.255092 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.255064    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dj667" event={"ID":"7dcf6319-3c75-4937-8f57-4c2735996721","Type":"ContainerStarted","Data":"bfc3ce865670ba43deb1832a8bf37d2306c9f0260d9d17591942209f7f324f64"}
Aug 04 03:09:46.697602 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.697028    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:46.697602 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:46.697189    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 03:09:46.697602 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:46.697250    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret podName:550248ce-69c9-4f77-9e9e-f6f0a509f0c4 nodeName:}" failed. No retries permitted until 2026-08-04 03:09:48.69723123 +0000 UTC m=+5.995371701 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret") pod "global-pull-secret-syncer-xbn5d" (UID: "550248ce-69c9-4f77-9e9e-f6f0a509f0c4") : object "kube-system"/"original-pull-secret" not registered
Aug 04 03:09:46.797557 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.797506    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:46.797760 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:46.797598    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77xt\" (UniqueName: \"kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt\") pod \"network-check-target-6lt8h\" (UID: \"409a5d1e-64ba-40a1-9bfb-57211e03d6ba\") " pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:46.797870 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:46.797765    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 03:09:46.797870 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:46.797786    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 03:09:46.797870 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:46.797799    2559 projected.go:196] Error preparing data for projected volume kube-api-access-s77xt for pod openshift-network-diagnostics/network-check-target-6lt8h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:09:46.797870 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:46.797860    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt podName:409a5d1e-64ba-40a1-9bfb-57211e03d6ba nodeName:}" failed. No retries permitted until 2026-08-04 03:09:48.797840514 +0000 UTC m=+6.095981069 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-s77xt" (UniqueName: "kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt") pod "network-check-target-6lt8h" (UID: "409a5d1e-64ba-40a1-9bfb-57211e03d6ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:09:46.798360 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:46.798240    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:09:46.798360 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:46.798303    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs podName:57750f19-0d04-432e-94f1-d0d0fde883bf nodeName:}" failed. No retries permitted until 2026-08-04 03:09:48.79828614 +0000 UTC m=+6.096426621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs") pod "network-metrics-daemon-xlcd8" (UID: "57750f19-0d04-432e-94f1-d0d0fde883bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:09:47.112408 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:47.112343    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 03:04:45 +0000 UTC" deadline="2028-02-07 11:47:10.882070791 +0000 UTC"
Aug 04 03:09:47.112408 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:47.112401    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13256h37m23.769673799s"
Aug 04 03:09:47.232725 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:47.232690    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:47.232887 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:47.232823    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:09:47.234619 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:47.234599    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:47.234816 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:47.234792    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:09:47.276904 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:47.276843    2559 generic.go:356] "Generic (PLEG): container finished" podID="314c6e474f45a46df975dbb8866f1f60" containerID="fa4e0d904f9d64d98a16f0480d9b4edc5461071b2f4d7be0bd1d92491e4c1478" exitCode=0
Aug 04 03:09:47.277960 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:47.277918    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal" event={"ID":"314c6e474f45a46df975dbb8866f1f60","Type":"ContainerDied","Data":"fa4e0d904f9d64d98a16f0480d9b4edc5461071b2f4d7be0bd1d92491e4c1478"}
Aug 04 03:09:47.294357 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:47.294291    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-110.ec2.internal" podStartSLOduration=3.294270502 podStartE2EDuration="3.294270502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:09:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:09:46.254855397 +0000 UTC m=+3.552995889" watchObservedRunningTime="2026-08-04 03:09:47.294270502 +0000 UTC m=+4.592411020"
Aug 04 03:09:48.232163 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:48.232130    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:48.232700 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:48.232257    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:09:48.304927 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:48.304888    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal" event={"ID":"314c6e474f45a46df975dbb8866f1f60","Type":"ContainerStarted","Data":"350199f3b9c2c57cb45f4b7fc3fb9ad994444b2a01d58c4d72c65b08c679d69a"}
Aug 04 03:09:48.719412 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:48.719323    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:48.719617 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:48.719506    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 03:09:48.719617 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:48.719595    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret podName:550248ce-69c9-4f77-9e9e-f6f0a509f0c4 nodeName:}" failed. No retries permitted until 2026-08-04 03:09:52.719571697 +0000 UTC m=+10.017712182 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret") pod "global-pull-secret-syncer-xbn5d" (UID: "550248ce-69c9-4f77-9e9e-f6f0a509f0c4") : object "kube-system"/"original-pull-secret" not registered
Aug 04 03:09:48.820641 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:48.820602    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:48.820799 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:48.820678    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77xt\" (UniqueName: \"kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt\") pod \"network-check-target-6lt8h\" (UID: \"409a5d1e-64ba-40a1-9bfb-57211e03d6ba\") " pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:48.820858 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:48.820821    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 03:09:48.820858 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:48.820840    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 03:09:48.820858 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:48.820853    2559 projected.go:196] Error preparing data for projected volume kube-api-access-s77xt for pod openshift-network-diagnostics/network-check-target-6lt8h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:09:48.821004 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:48.820911    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt podName:409a5d1e-64ba-40a1-9bfb-57211e03d6ba nodeName:}" failed. No retries permitted until 2026-08-04 03:09:52.820890412 +0000 UTC m=+10.119030880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-s77xt" (UniqueName: "kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt") pod "network-check-target-6lt8h" (UID: "409a5d1e-64ba-40a1-9bfb-57211e03d6ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:09:48.821335 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:48.821315    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:09:48.821441 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:48.821370    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs podName:57750f19-0d04-432e-94f1-d0d0fde883bf nodeName:}" failed. No retries permitted until 2026-08-04 03:09:52.821354185 +0000 UTC m=+10.119494651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs") pod "network-metrics-daemon-xlcd8" (UID: "57750f19-0d04-432e-94f1-d0d0fde883bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:09:49.232975 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:49.232942    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:49.233435 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:49.232991    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:49.233435 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:49.233091    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:09:49.233435 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:49.233224    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:09:50.232206 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:50.231928    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:50.232206 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:50.232076    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:09:51.232687 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:51.232158    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:51.232687 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:51.232294    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:09:51.233437 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:51.233292    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:51.233437 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:51.233420    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:09:52.232535 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:52.232500    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:52.232721 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:52.232641    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:09:52.752781 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:52.752743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:52.752945 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:52.752903    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 03:09:52.753005 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:52.752971    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret podName:550248ce-69c9-4f77-9e9e-f6f0a509f0c4 nodeName:}" failed. No retries permitted until 2026-08-04 03:10:00.752951126 +0000 UTC m=+18.051091607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret") pod "global-pull-secret-syncer-xbn5d" (UID: "550248ce-69c9-4f77-9e9e-f6f0a509f0c4") : object "kube-system"/"original-pull-secret" not registered
Aug 04 03:09:52.853621 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:52.853562    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:52.853810 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:52.853635    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77xt\" (UniqueName: \"kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt\") pod \"network-check-target-6lt8h\" (UID: \"409a5d1e-64ba-40a1-9bfb-57211e03d6ba\") " pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:52.853810 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:52.853780    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:09:52.853919 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:52.853851    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs podName:57750f19-0d04-432e-94f1-d0d0fde883bf nodeName:}" failed. No retries permitted until 2026-08-04 03:10:00.853831202 +0000 UTC m=+18.151971668 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs") pod "network-metrics-daemon-xlcd8" (UID: "57750f19-0d04-432e-94f1-d0d0fde883bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:09:52.853919 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:52.853852    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 03:09:52.853919 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:52.853875    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 03:09:52.853919 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:52.853889    2559 projected.go:196] Error preparing data for projected volume kube-api-access-s77xt for pod openshift-network-diagnostics/network-check-target-6lt8h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:09:52.854112 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:52.853936    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt podName:409a5d1e-64ba-40a1-9bfb-57211e03d6ba nodeName:}" failed. No retries permitted until 2026-08-04 03:10:00.853921237 +0000 UTC m=+18.152061706 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-s77xt" (UniqueName: "kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt") pod "network-check-target-6lt8h" (UID: "409a5d1e-64ba-40a1-9bfb-57211e03d6ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:09:53.232518 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:53.232436    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:53.232693 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:53.232592    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:09:53.232693 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:53.232616    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:53.233402 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:53.233349    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:09:54.232285 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:54.232197    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:54.232479 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:54.232333    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:09:55.232425 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:55.232388    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:55.232964 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:55.232516    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:09:55.232964 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:55.232389    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:55.232964 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:55.232697    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:09:56.232312 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:56.232277    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:56.232506 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:56.232403    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:09:57.232618 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:57.232580    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:57.233055 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:57.232583    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:57.233055 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:57.232732    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:09:57.233055 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:57.232814    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:09:58.232324 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:58.232291    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:09:58.232527 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:58.232398    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:09:59.232491 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:59.232455    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:09:59.232956 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:09:59.232465    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:09:59.232956 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:59.232615    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:09:59.232956 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:09:59.232667    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:10:00.232693 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:00.232659    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:00.233220 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:00.232783    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:10:00.817373 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:00.817330    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:00.817532 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:00.817473    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 03:10:00.817587 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:00.817536    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret podName:550248ce-69c9-4f77-9e9e-f6f0a509f0c4 nodeName:}" failed. No retries permitted until 2026-08-04 03:10:16.81752032 +0000 UTC m=+34.115660786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret") pod "global-pull-secret-syncer-xbn5d" (UID: "550248ce-69c9-4f77-9e9e-f6f0a509f0c4") : object "kube-system"/"original-pull-secret" not registered
Aug 04 03:10:00.918431 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:00.918394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77xt\" (UniqueName: \"kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt\") pod \"network-check-target-6lt8h\" (UID: \"409a5d1e-64ba-40a1-9bfb-57211e03d6ba\") " pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:00.918656 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:00.918469    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:00.918656 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:00.918602    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:10:00.918656 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:00.918617    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 03:10:00.918656 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:00.918644    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 03:10:00.918656 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:00.918657    2559 projected.go:196] Error preparing data for projected volume kube-api-access-s77xt for pod openshift-network-diagnostics/network-check-target-6lt8h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:10:00.918851 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:00.918663    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs podName:57750f19-0d04-432e-94f1-d0d0fde883bf nodeName:}" failed. No retries permitted until 2026-08-04 03:10:16.91864436 +0000 UTC m=+34.216784846 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs") pod "network-metrics-daemon-xlcd8" (UID: "57750f19-0d04-432e-94f1-d0d0fde883bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:10:00.918851 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:00.918707    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt podName:409a5d1e-64ba-40a1-9bfb-57211e03d6ba nodeName:}" failed. No retries permitted until 2026-08-04 03:10:16.918691154 +0000 UTC m=+34.216831643 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-s77xt" (UniqueName: "kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt") pod "network-check-target-6lt8h" (UID: "409a5d1e-64ba-40a1-9bfb-57211e03d6ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:10:01.231966 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:01.231883    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:01.232134 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:01.231883    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:01.232134 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:01.232025    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:10:01.232256 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:01.232128    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:10:02.232233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:02.232203    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:02.232729 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:02.232334    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:10:03.232981 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:03.232939    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:03.233396 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:03.233067    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:10:03.233396 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:03.233122    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:03.233396 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:03.233238    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:10:04.231937 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:04.231906    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:04.232109 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:04.232021    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:10:05.232327 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.232048    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:05.232800 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.232048    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:05.232800 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:05.232427    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:10:05.232800 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:05.232471    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:10:05.348579 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.348475    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw" event={"ID":"842cb690-dfeb-49ec-aef1-401993115383","Type":"ContainerStarted","Data":"4b6d95032b8f9997e0d77ac467c27d1ca47e3ef1a99884a6214515824cbd874a"}
Aug 04 03:10:05.349956 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.349930    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f4mrm" event={"ID":"6864c31c-d3e5-4224-beda-e85b0d20868d","Type":"ContainerStarted","Data":"dc32b45f2b6d2e8e1ce8dea0aa0e0e10595a387dcceac0ebd8ebfc8aa2acccd1"}
Aug 04 03:10:05.352886 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.352865    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs" event={"ID":"1e99772e-31e2-4eda-88af-0fc77fb0156e","Type":"ContainerStarted","Data":"9008e4752ee0161d5c65dfcb8e9c001bcf42b5843b8af74f31f0324078ddcb76"}
Aug 04 03:10:05.352987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.352931    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs" event={"ID":"1e99772e-31e2-4eda-88af-0fc77fb0156e","Type":"ContainerStarted","Data":"64b650aa16b2cf64f401441236410107eba248487cb347e14450526d694a07fa"}
Aug 04 03:10:05.352987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.352979    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs" event={"ID":"1e99772e-31e2-4eda-88af-0fc77fb0156e","Type":"ContainerStarted","Data":"4e456670eb2ebe2c357d2cbc08dd93dc9947bcf1d003217493d8d6f2842ef358"}
Aug 04 03:10:05.353097 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.353032    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs" event={"ID":"1e99772e-31e2-4eda-88af-0fc77fb0156e","Type":"ContainerStarted","Data":"9577ed167b82929762e9947efca1511afcf7226edfb903edcfedc5d8753374a6"}
Aug 04 03:10:05.353097 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.353067    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs" event={"ID":"1e99772e-31e2-4eda-88af-0fc77fb0156e","Type":"ContainerStarted","Data":"b354076bcafbc09acbed4151ad192ac8de28f9d10fc1a2e3183b6d48f7493a23"}
Aug 04 03:10:05.353195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.353106    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs" event={"ID":"1e99772e-31e2-4eda-88af-0fc77fb0156e","Type":"ContainerStarted","Data":"a133a2be538719853e1b2320dcbaad698a80543737036484c3c2760d90565239"}
Aug 04 03:10:05.354264 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.354245    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dj667" event={"ID":"7dcf6319-3c75-4937-8f57-4c2735996721","Type":"ContainerStarted","Data":"19a073af690cd61a8cee5387ecf67269e61356112402484c614e124ee74f7e1b"}
Aug 04 03:10:05.355642 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.355627    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2svj7" event={"ID":"48fcbb9a-7f25-411d-834f-00594bc5fb4f","Type":"ContainerStarted","Data":"1cfa83a9dfc85a115e8520d0eb1f84306bde66fac638d799dbb4075f8327e90a"}
Aug 04 03:10:05.356964 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.356944    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6nqfs" event={"ID":"bbc9759d-10ad-4442-9a87-868e5ba4749a","Type":"ContainerStarted","Data":"12b47e50b7fb0d6a9881c296bb72a875315ebba5458fb109805243ec5d1490c4"}
Aug 04 03:10:05.358357 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.358335    2559 generic.go:356] "Generic (PLEG): container finished" podID="df20bbca-eb3a-44c3-be85-289103e0f04d" containerID="b375367bffc4b692cc8aa016ad19578a2e1fcdab16e81ffae5745dd0e09222c7" exitCode=0
Aug 04 03:10:05.358475 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.358413    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dn58s" event={"ID":"df20bbca-eb3a-44c3-be85-289103e0f04d","Type":"ContainerDied","Data":"b375367bffc4b692cc8aa016ad19578a2e1fcdab16e81ffae5745dd0e09222c7"}
Aug 04 03:10:05.359919 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.359900    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9m9ks" event={"ID":"7a2ea3c8-88ca-4802-987b-be5942899884","Type":"ContainerStarted","Data":"dbaa8a4a388aa5a456d6ea649add0054d43f637a84eeece0d6a3f433f84595e9"}
Aug 04 03:10:05.363926 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.363888    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-f4mrm" podStartSLOduration=3.924203875 podStartE2EDuration="22.363876725s" totalImagesPullingTime="18.43967285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:05.363462496 +0000 UTC m=+22.661603004" watchObservedRunningTime="2026-08-04 03:10:05.363876725 +0000 UTC m=+22.662017213"
Aug 04 03:10:05.364006 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.363954    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-110.ec2.internal" podStartSLOduration=21.363949012 podStartE2EDuration="21.363949012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:09:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:09:48.320389389 +0000 UTC m=+5.618529878" watchObservedRunningTime="2026-08-04 03:10:05.363949012 +0000 UTC m=+22.662089555"
Aug 04 03:10:05.399353 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.399310    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2svj7" podStartSLOduration=3.95656375 podStartE2EDuration="22.399296777s" totalImagesPullingTime="18.442733027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:05.399128463 +0000 UTC m=+22.697268950" watchObservedRunningTime="2026-08-04 03:10:05.399296777 +0000 UTC m=+22.697437266"
Aug 04 03:10:05.416913 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.416874    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6nqfs" podStartSLOduration=3.94078669 podStartE2EDuration="22.416855577s" totalImagesPullingTime="18.476068887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:05.416757641 +0000 UTC m=+22.714898131" watchObservedRunningTime="2026-08-04 03:10:05.416855577 +0000 UTC m=+22.714996065"
Aug 04 03:10:05.433150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.433109    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dj667" podStartSLOduration=3.914337985 podStartE2EDuration="22.433095004s" totalImagesPullingTime="18.518757019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:05.43263065 +0000 UTC m=+22.730771159" watchObservedRunningTime="2026-08-04 03:10:05.433095004 +0000 UTC m=+22.731235493"
Aug 04 03:10:05.449445 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.449403    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9m9ks" podStartSLOduration=11.87300016 podStartE2EDuration="22.449389971s" totalImagesPullingTime="10.576389811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:05.449011525 +0000 UTC m=+22.747152015" watchObservedRunningTime="2026-08-04 03:10:05.449389971 +0000 UTC m=+22.747530500"
Aug 04 03:10:05.468666 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.468632    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:05.549971 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.549934    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:05.652999 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:05.652966    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:06.232277 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:06.232206    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:06.232450 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:06.232317    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:10:06.364576 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:06.364481    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rgclr" event={"ID":"e1ea832d-816f-48c5-b6aa-1cefcae84ebe","Type":"ContainerStarted","Data":"2340ffd79812b35131d6f44c43e941f254e07891cc51b08b17b9917f316eb9c0"}
Aug 04 03:10:06.377569 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:06.377506    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rgclr" podStartSLOduration=4.92821852 podStartE2EDuration="23.377490484s" totalImagesPullingTime="18.449271964s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:06.377365795 +0000 UTC m=+23.675506306" watchObservedRunningTime="2026-08-04 03:10:06.377490484 +0000 UTC m=+23.675630973"
Aug 04 03:10:06.406931 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:06.406897    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 04 03:10:06.496386 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:06.496299    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:06.552719 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:06.552685    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:07.157132 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:07.157021    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T03:10:06.406922968Z","UUID":"94b9fcab-3521-41f2-93d7-cbcbe9b903b4","Handler":null,"Name":"","Endpoint":""}
Aug 04 03:10:07.159271 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:07.158996    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 04 03:10:07.159271 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:07.159053    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 04 03:10:07.235794 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:07.235769    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:07.236248 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:07.235898    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:10:07.236248 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:07.235952    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:07.236248 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:07.236067    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:10:07.369250 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:07.369217    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw" event={"ID":"842cb690-dfeb-49ec-aef1-401993115383","Type":"ContainerStarted","Data":"c8a34214bc9d327b451b1cb4b2f1b250eac08dca97bb30d8224d06abfb791852"}
Aug 04 03:10:08.152569 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:08.152463    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:08.232530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:08.232495    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:08.232706 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:08.232617    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:10:08.252716 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:08.252654    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:08.373919 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:08.373879    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw" event={"ID":"842cb690-dfeb-49ec-aef1-401993115383","Type":"ContainerStarted","Data":"5be1ae35f47bc5151053f32286fe9aa5b343f3652f5a187c9813c97ee679fab8"}
Aug 04 03:10:08.377597 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:08.377570    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs" event={"ID":"1e99772e-31e2-4eda-88af-0fc77fb0156e","Type":"ContainerStarted","Data":"8bf7ba423c886f3398c12e49e9a80c4990473719e734407bf07c6ff95b70c135"}
Aug 04 03:10:08.390754 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:08.390703    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8jsrw" podStartSLOduration=4.083423183 podStartE2EDuration="25.390685056s" totalImagesPullingTime="21.307261873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:08.390577496 +0000 UTC m=+25.688718007" watchObservedRunningTime="2026-08-04 03:10:08.390685056 +0000 UTC m=+25.688825544"
Aug 04 03:10:08.446048 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:08.445759    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:10:08.446599 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:08.446580    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:10:09.232007 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:09.231965    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:09.232175 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:09.232125    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:10:09.232175 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:09.232148    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:09.232298 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:09.232276    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:10:09.380307 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:09.380186    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:10:09.380770 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:09.380621    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2svj7"
Aug 04 03:10:09.381474 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:09.381446    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:09.483179 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:09.483105    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:09.583846 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:09.583813    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:10.232469 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:10.232442    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:10.232626 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:10.232554    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:10:10.385215 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:10.385172    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs" event={"ID":"1e99772e-31e2-4eda-88af-0fc77fb0156e","Type":"ContainerStarted","Data":"5d6b40a46e61fc9db54796cf8da0f4538259aaa79d28a5e555c50c6eb403c425"}
Aug 04 03:10:10.386184 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:10.385353    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:10:10.386184 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:10.385493    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:10:10.387121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:10.387081    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dn58s" event={"ID":"df20bbca-eb3a-44c3-be85-289103e0f04d","Type":"ContainerStarted","Data":"0dfafce7f9d05f6b2ae1198ebef6921e3213ef1a6883b4543ed90383fc3571aa"}
Aug 04 03:10:10.400701 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:10.400633    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:10:10.400808 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:10.400739    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:10:10.412123 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:10.412080    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs" podStartSLOduration=8.907951163 podStartE2EDuration="27.412065209s" totalImagesPullingTime="18.504114046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:10.410789619 +0000 UTC m=+27.708930107" watchObservedRunningTime="2026-08-04 03:10:10.412065209 +0000 UTC m=+27.710205697"
Aug 04 03:10:11.105972 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:11.105937    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:10:11.233002 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:11.232675    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:11.233002 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:11.232818    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:10:11.233647 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:11.233629    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:11.233889 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:11.233869    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:10:11.390886 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:11.390848    2559 generic.go:356] "Generic (PLEG): container finished" podID="df20bbca-eb3a-44c3-be85-289103e0f04d" containerID="0dfafce7f9d05f6b2ae1198ebef6921e3213ef1a6883b4543ed90383fc3571aa" exitCode=0
Aug 04 03:10:11.391305 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:11.390941    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dn58s" event={"ID":"df20bbca-eb3a-44c3-be85-289103e0f04d","Type":"ContainerDied","Data":"0dfafce7f9d05f6b2ae1198ebef6921e3213ef1a6883b4543ed90383fc3571aa"}
Aug 04 03:10:12.148659 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:12.148402    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xlcd8"]
Aug 04 03:10:12.148836 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:12.148752    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:12.148888 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:12.148851    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:10:12.149873 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:12.149828    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xbn5d"]
Aug 04 03:10:12.150045 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:12.149933    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:12.150045 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:12.150035    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:10:12.150675 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:12.150650    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6lt8h"]
Aug 04 03:10:12.150792 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:12.150755    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:12.150867 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:12.150844    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:10:12.857201 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:12.857122    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:13.233178 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:13.233093    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:13.233316 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:13.233191    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:10:13.397913 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:13.397873    2559 generic.go:356] "Generic (PLEG): container finished" podID="df20bbca-eb3a-44c3-be85-289103e0f04d" containerID="2e6032584f4e0ef2a43f8528f9d64c235c22ef4fcfde30e6d07b823a3f67d6ea" exitCode=0
Aug 04 03:10:13.398074 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:13.397935    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dn58s" event={"ID":"df20bbca-eb3a-44c3-be85-289103e0f04d","Type":"ContainerDied","Data":"2e6032584f4e0ef2a43f8528f9d64c235c22ef4fcfde30e6d07b823a3f67d6ea"}
Aug 04 03:10:14.231916 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:14.231837    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:14.232252 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:14.231837    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:14.232252 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:14.231944    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:10:14.232252 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:14.232014    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:10:15.232097 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:15.231922    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:15.232590 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:15.232172    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:10:15.403801 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:15.403767    2559 generic.go:356] "Generic (PLEG): container finished" podID="df20bbca-eb3a-44c3-be85-289103e0f04d" containerID="8aa0e230b6c50e2ee17d2420e1b0a7350459bbcbb963cf4af7337d0f29eaddc2" exitCode=0
Aug 04 03:10:15.403960 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:15.403822    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dn58s" event={"ID":"df20bbca-eb3a-44c3-be85-289103e0f04d","Type":"ContainerDied","Data":"8aa0e230b6c50e2ee17d2420e1b0a7350459bbcbb963cf4af7337d0f29eaddc2"}
Aug 04 03:10:15.817485 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:15.817446    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:15.913189 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:15.913150    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:15.964620 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:15.964587    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:16.232747 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:16.232665    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:16.233195 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:16.232779    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6lt8h" podUID="409a5d1e-64ba-40a1-9bfb-57211e03d6ba"
Aug 04 03:10:16.233195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:16.232843    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:16.233195 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:16.232943    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbn5d" podUID="550248ce-69c9-4f77-9e9e-f6f0a509f0c4"
Aug 04 03:10:16.379665 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:16.379616    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:16.481272 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:16.481239    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:16.835124 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:16.835089    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:16.835319 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:16.835247    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 03:10:16.835319 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:16.835317    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret podName:550248ce-69c9-4f77-9e9e-f6f0a509f0c4 nodeName:}" failed. No retries permitted until 2026-08-04 03:10:48.835296585 +0000 UTC m=+66.133437062 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret") pod "global-pull-secret-syncer-xbn5d" (UID: "550248ce-69c9-4f77-9e9e-f6f0a509f0c4") : object "kube-system"/"original-pull-secret" not registered
Aug 04 03:10:16.935474 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:16.935442    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77xt\" (UniqueName: \"kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt\") pod \"network-check-target-6lt8h\" (UID: \"409a5d1e-64ba-40a1-9bfb-57211e03d6ba\") " pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:16.935654 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:16.935498    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:16.935654 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:16.935621    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:10:16.935747 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:16.935653    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 03:10:16.935747 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:16.935681    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs podName:57750f19-0d04-432e-94f1-d0d0fde883bf nodeName:}" failed. No retries permitted until 2026-08-04 03:10:48.935660619 +0000 UTC m=+66.233801084 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs") pod "network-metrics-daemon-xlcd8" (UID: "57750f19-0d04-432e-94f1-d0d0fde883bf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 03:10:16.935747 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:16.935694    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 03:10:16.935747 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:16.935710    2559 projected.go:196] Error preparing data for projected volume kube-api-access-s77xt for pod openshift-network-diagnostics/network-check-target-6lt8h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:10:16.935879 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:16.935753    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt podName:409a5d1e-64ba-40a1-9bfb-57211e03d6ba nodeName:}" failed. No retries permitted until 2026-08-04 03:10:48.935741945 +0000 UTC m=+66.233882410 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-s77xt" (UniqueName: "kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt") pod "network-check-target-6lt8h" (UID: "409a5d1e-64ba-40a1-9bfb-57211e03d6ba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 03:10:17.231951 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.231922    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:17.232134 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:17.232058    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:10:17.526988 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.526913    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-110.ec2.internal" event="NodeReady"
Aug 04 03:10:17.527426 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.527062    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 04 03:10:17.570552 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.570516    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-78fkp"]
Aug 04 03:10:17.572636 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.572617    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xqt4c"]
Aug 04 03:10:17.572765 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.572749    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.574168 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.574151    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xqt4c"
Aug 04 03:10:17.574942 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.574920    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 04 03:10:17.575038 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.574931    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 04 03:10:17.575038 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.575023    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4d7vw\""
Aug 04 03:10:17.576027 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.576010    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 04 03:10:17.576297 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.576280    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 04 03:10:17.576444 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.576424    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 04 03:10:17.576656 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.576643    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xvw8f\""
Aug 04 03:10:17.580615 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.580595    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xqt4c"]
Aug 04 03:10:17.583976 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.583925    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-78fkp"]
Aug 04 03:10:17.712974 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.712938    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:17.741794 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.741762    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hcj6\" (UniqueName: \"kubernetes.io/projected/5eb1ab5d-b609-4822-baba-4357e434097f-kube-api-access-4hcj6\") pod \"ingress-canary-xqt4c\" (UID: \"5eb1ab5d-b609-4822-baba-4357e434097f\") " pod="openshift-ingress-canary/ingress-canary-xqt4c"
Aug 04 03:10:17.741794 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.741794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-config-volume\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.742027 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.741818    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz7hn\" (UniqueName: \"kubernetes.io/projected/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-kube-api-access-xz7hn\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.742027 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.741891    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5eb1ab5d-b609-4822-baba-4357e434097f-cert\") pod \"ingress-canary-xqt4c\" (UID: \"5eb1ab5d-b609-4822-baba-4357e434097f\") " pod="openshift-ingress-canary/ingress-canary-xqt4c"
Aug 04 03:10:17.742027 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.741937    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-tmp-dir\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.742027 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.741969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.764048 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.764018    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:17.842603 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.842513    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.842603 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.842589    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hcj6\" (UniqueName: \"kubernetes.io/projected/5eb1ab5d-b609-4822-baba-4357e434097f-kube-api-access-4hcj6\") pod \"ingress-canary-xqt4c\" (UID: \"5eb1ab5d-b609-4822-baba-4357e434097f\") " pod="openshift-ingress-canary/ingress-canary-xqt4c"
Aug 04 03:10:17.842835 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.842612    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-config-volume\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.842835 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.842643    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz7hn\" (UniqueName: \"kubernetes.io/projected/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-kube-api-access-xz7hn\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.842835 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:17.842671    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 03:10:17.842835 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.842685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5eb1ab5d-b609-4822-baba-4357e434097f-cert\") pod \"ingress-canary-xqt4c\" (UID: \"5eb1ab5d-b609-4822-baba-4357e434097f\") " pod="openshift-ingress-canary/ingress-canary-xqt4c"
Aug 04 03:10:17.842835 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:17.842750    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls podName:a98e852a-83ee-40a7-8cc0-7243fcf6bafb nodeName:}" failed. No retries permitted until 2026-08-04 03:10:18.34273052 +0000 UTC m=+35.640870986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls") pod "dns-default-78fkp" (UID: "a98e852a-83ee-40a7-8cc0-7243fcf6bafb") : secret "dns-default-metrics-tls" not found
Aug 04 03:10:17.842835 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.842773    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-tmp-dir\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.843132 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.843117    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-tmp-dir\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.843626 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.843571    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-config-volume\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.848070 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.848045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5eb1ab5d-b609-4822-baba-4357e434097f-cert\") pod \"ingress-canary-xqt4c\" (UID: \"5eb1ab5d-b609-4822-baba-4357e434097f\") " pod="openshift-ingress-canary/ingress-canary-xqt4c"
Aug 04 03:10:17.851451 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.851407    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz7hn\" (UniqueName: \"kubernetes.io/projected/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-kube-api-access-xz7hn\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:17.851599 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.851580    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hcj6\" (UniqueName: \"kubernetes.io/projected/5eb1ab5d-b609-4822-baba-4357e434097f-kube-api-access-4hcj6\") pod \"ingress-canary-xqt4c\" (UID: \"5eb1ab5d-b609-4822-baba-4357e434097f\") " pod="openshift-ingress-canary/ingress-canary-xqt4c"
Aug 04 03:10:17.892620 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:17.892585    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xqt4c"
Aug 04 03:10:18.232172 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:18.232087    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:18.232342 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:18.232088    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:18.235118 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:18.235089    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 03:10:18.235642 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:18.235414    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jb5ws\""
Aug 04 03:10:18.235642 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:18.235419    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 03:10:18.235817 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:18.235759    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 03:10:18.346516 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:18.346486    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:18.346696 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:18.346635    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 03:10:18.346743 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:18.346703    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls podName:a98e852a-83ee-40a7-8cc0-7243fcf6bafb nodeName:}" failed. No retries permitted until 2026-08-04 03:10:19.34667901 +0000 UTC m=+36.644819479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls") pod "dns-default-78fkp" (UID: "a98e852a-83ee-40a7-8cc0-7243fcf6bafb") : secret "dns-default-metrics-tls" not found
Aug 04 03:10:19.232184 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:19.232153    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:19.237906 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:19.237618    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6l47c\""
Aug 04 03:10:19.237906 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:19.237673    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 03:10:19.342959 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:19.342798    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xqt4c"]
Aug 04 03:10:19.345993 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:10:19.345963    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb1ab5d_b609_4822_baba_4357e434097f.slice/crio-801a780f7d9fa157dc2bf6412933ea89b0a711f01505e323597da07390b6503b WatchSource:0}: Error finding container 801a780f7d9fa157dc2bf6412933ea89b0a711f01505e323597da07390b6503b: Status 404 returned error can't find the container with id 801a780f7d9fa157dc2bf6412933ea89b0a711f01505e323597da07390b6503b
Aug 04 03:10:19.355240 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:19.355205    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:19.355367 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:19.355348    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 03:10:19.355427 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:19.355404    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls podName:a98e852a-83ee-40a7-8cc0-7243fcf6bafb nodeName:}" failed. No retries permitted until 2026-08-04 03:10:21.355389603 +0000 UTC m=+38.653530069 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls") pod "dns-default-78fkp" (UID: "a98e852a-83ee-40a7-8cc0-7243fcf6bafb") : secret "dns-default-metrics-tls" not found
Aug 04 03:10:19.416531 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:19.416498    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dn58s" event={"ID":"df20bbca-eb3a-44c3-be85-289103e0f04d","Type":"ContainerStarted","Data":"961fbb58818adaf3a1a894dcee9bcb56643e5ba8107995e23c6dea42775c9067"}
Aug 04 03:10:19.417696 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:19.417673    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xqt4c" event={"ID":"5eb1ab5d-b609-4822-baba-4357e434097f","Type":"ContainerStarted","Data":"801a780f7d9fa157dc2bf6412933ea89b0a711f01505e323597da07390b6503b"}
Aug 04 03:10:20.422996 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:20.422965    2559 generic.go:356] "Generic (PLEG): container finished" podID="df20bbca-eb3a-44c3-be85-289103e0f04d" containerID="961fbb58818adaf3a1a894dcee9bcb56643e5ba8107995e23c6dea42775c9067" exitCode=0
Aug 04 03:10:20.423462 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:20.423027    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dn58s" event={"ID":"df20bbca-eb3a-44c3-be85-289103e0f04d","Type":"ContainerDied","Data":"961fbb58818adaf3a1a894dcee9bcb56643e5ba8107995e23c6dea42775c9067"}
Aug 04 03:10:21.370877 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:21.370844    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:21.371030 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:21.370997    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 03:10:21.371073 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:21.371063    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls podName:a98e852a-83ee-40a7-8cc0-7243fcf6bafb nodeName:}" failed. No retries permitted until 2026-08-04 03:10:25.371047073 +0000 UTC m=+42.669187540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls") pod "dns-default-78fkp" (UID: "a98e852a-83ee-40a7-8cc0-7243fcf6bafb") : secret "dns-default-metrics-tls" not found
Aug 04 03:10:21.427827 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:21.427797    2559 generic.go:356] "Generic (PLEG): container finished" podID="df20bbca-eb3a-44c3-be85-289103e0f04d" containerID="d631ade98b8f8e9adf06dab375fc051669e62fd9ee744614c141fa3756b649ca" exitCode=0
Aug 04 03:10:21.428181 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:21.427836    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dn58s" event={"ID":"df20bbca-eb3a-44c3-be85-289103e0f04d","Type":"ContainerDied","Data":"d631ade98b8f8e9adf06dab375fc051669e62fd9ee744614c141fa3756b649ca"}
Aug 04 03:10:21.778171 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:21.778143    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:21.879020 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:21.878987    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:22.435984 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:22.435948    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dn58s" event={"ID":"df20bbca-eb3a-44c3-be85-289103e0f04d","Type":"ContainerStarted","Data":"f37fd5a11a08e490a16b999fec1e14c720dbc02193d052e748117842a22f204c"}
Aug 04 03:10:22.437302 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:22.437279    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xqt4c" event={"ID":"5eb1ab5d-b609-4822-baba-4357e434097f","Type":"ContainerStarted","Data":"86547a7262d6e0a6739e3f5e8ad9ba22806bb7bdb2fbb52bb63f6a152a34f8d8"}
Aug 04 03:10:22.456828 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:22.456773    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dn58s" podStartSLOduration=10.301581014 podStartE2EDuration="39.456758919s" totalImagesPullingTime="29.155177905s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:22.455321001 +0000 UTC m=+39.753461506" watchObservedRunningTime="2026-08-04 03:10:22.456758919 +0000 UTC m=+39.754899417"
Aug 04 03:10:22.468978 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:22.468930    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xqt4c" podStartSLOduration=3.114324517 podStartE2EDuration="5.468913372s" totalImagesPullingTime="2.354588855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:22.468099076 +0000 UTC m=+39.766239563" watchObservedRunningTime="2026-08-04 03:10:22.468913372 +0000 UTC m=+39.767053860"
Aug 04 03:10:22.545180 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:22.545143    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:22.605450 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:22.605418    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:22.675002 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:22.674973    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:25.398754 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:25.398711    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:25.399142 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:25.398842    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 03:10:25.399142 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:25.398895    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls podName:a98e852a-83ee-40a7-8cc0-7243fcf6bafb nodeName:}" failed. No retries permitted until 2026-08-04 03:10:33.398880699 +0000 UTC m=+50.697021166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls") pod "dns-default-78fkp" (UID: "a98e852a-83ee-40a7-8cc0-7243fcf6bafb") : secret "dns-default-metrics-tls" not found
Aug 04 03:10:30.151843 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:30.151806    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:30.210089 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:30.210055    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:33.457850 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:33.457818    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:33.458225 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:33.457964    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 03:10:33.458225 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:33.458019    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls podName:a98e852a-83ee-40a7-8cc0-7243fcf6bafb nodeName:}" failed. No retries permitted until 2026-08-04 03:10:49.458003633 +0000 UTC m=+66.756144099 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls") pod "dns-default-78fkp" (UID: "a98e852a-83ee-40a7-8cc0-7243fcf6bafb") : secret "dns-default-metrics-tls" not found
Aug 04 03:10:37.798970 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:37.798935    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:37.853057 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:37.853018    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:42.403949 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:42.403923    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kd7zs"
Aug 04 03:10:42.542650 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:42.542620    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:42.601891 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:42.601861    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:10:48.870001 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:48.869953    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:48.872810 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:48.872790    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 03:10:48.883966 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:48.883941    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/550248ce-69c9-4f77-9e9e-f6f0a509f0c4-original-pull-secret\") pod \"global-pull-secret-syncer-xbn5d\" (UID: \"550248ce-69c9-4f77-9e9e-f6f0a509f0c4\") " pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:48.970504 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:48.970457    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:10:48.970682 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:48.970530    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77xt\" (UniqueName: \"kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt\") pod \"network-check-target-6lt8h\" (UID: \"409a5d1e-64ba-40a1-9bfb-57211e03d6ba\") " pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:48.973145 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:48.973126    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 03:10:48.973236 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:48.973170    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 03:10:48.981125 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:48.981098    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 04 03:10:48.981211 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:48.981175    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs podName:57750f19-0d04-432e-94f1-d0d0fde883bf nodeName:}" failed. No retries permitted until 2026-08-04 03:11:52.981151582 +0000 UTC m=+130.279292068 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs") pod "network-metrics-daemon-xlcd8" (UID: "57750f19-0d04-432e-94f1-d0d0fde883bf") : secret "metrics-daemon-secret" not found
Aug 04 03:10:48.983145 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:48.983129    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 03:10:48.994183 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:48.994160    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s77xt\" (UniqueName: \"kubernetes.io/projected/409a5d1e-64ba-40a1-9bfb-57211e03d6ba-kube-api-access-s77xt\") pod \"network-check-target-6lt8h\" (UID: \"409a5d1e-64ba-40a1-9bfb-57211e03d6ba\") " pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:49.147625 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:49.147529    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jb5ws\""
Aug 04 03:10:49.149241 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:49.149223    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbn5d"
Aug 04 03:10:49.155864 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:49.155844    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:49.278258 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:49.278227    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xbn5d"]
Aug 04 03:10:49.281580 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:10:49.281553    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod550248ce_69c9_4f77_9e9e_f6f0a509f0c4.slice/crio-f9b758ff1ce4f7a09d2a06496d18660902c29f38e6986ccccfafc1417cfbb615 WatchSource:0}: Error finding container f9b758ff1ce4f7a09d2a06496d18660902c29f38e6986ccccfafc1417cfbb615: Status 404 returned error can't find the container with id f9b758ff1ce4f7a09d2a06496d18660902c29f38e6986ccccfafc1417cfbb615
Aug 04 03:10:49.293398 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:49.293374    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6lt8h"]
Aug 04 03:10:49.295979 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:10:49.295954    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod409a5d1e_64ba_40a1_9bfb_57211e03d6ba.slice/crio-823e9068cc0263d3452a87cfd1047ef9f936c8fa4ad177c0132f049288ad1ef6 WatchSource:0}: Error finding container 823e9068cc0263d3452a87cfd1047ef9f936c8fa4ad177c0132f049288ad1ef6: Status 404 returned error can't find the container with id 823e9068cc0263d3452a87cfd1047ef9f936c8fa4ad177c0132f049288ad1ef6
Aug 04 03:10:49.474737 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:49.474645    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:10:49.474884 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:49.474760    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 03:10:49.474884 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:10:49.474809    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls podName:a98e852a-83ee-40a7-8cc0-7243fcf6bafb nodeName:}" failed. No retries permitted until 2026-08-04 03:11:21.474796178 +0000 UTC m=+98.772936644 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls") pod "dns-default-78fkp" (UID: "a98e852a-83ee-40a7-8cc0-7243fcf6bafb") : secret "dns-default-metrics-tls" not found
Aug 04 03:10:49.499849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:49.499815    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6lt8h" event={"ID":"409a5d1e-64ba-40a1-9bfb-57211e03d6ba","Type":"ContainerStarted","Data":"823e9068cc0263d3452a87cfd1047ef9f936c8fa4ad177c0132f049288ad1ef6"}
Aug 04 03:10:49.500851 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:49.500827    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xbn5d" event={"ID":"550248ce-69c9-4f77-9e9e-f6f0a509f0c4","Type":"ContainerStarted","Data":"f9b758ff1ce4f7a09d2a06496d18660902c29f38e6986ccccfafc1417cfbb615"}
Aug 04 03:10:54.514646 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:54.514599    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6lt8h" event={"ID":"409a5d1e-64ba-40a1-9bfb-57211e03d6ba","Type":"ContainerStarted","Data":"ba8fbfc61f4f4abe06eb68eb245bbbf68b2d2c83bcc45228485f570af1fd4357"}
Aug 04 03:10:54.515136 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:54.514696    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:10:54.515906 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:54.515884    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xbn5d" event={"ID":"550248ce-69c9-4f77-9e9e-f6f0a509f0c4","Type":"ContainerStarted","Data":"51599df19a5a2ec46468adf3d72997e2e10a9a7fa0644b7e41bbae691279488f"}
Aug 04 03:10:54.536183 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:10:54.536136    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6lt8h" podStartSLOduration=67.039951087 podStartE2EDuration="1m11.536122169s" totalImagesPullingTime="4.496171082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:54.535574168 +0000 UTC m=+71.833714653" watchObservedRunningTime="2026-08-04 03:10:54.536122169 +0000 UTC m=+71.834262656"
Aug 04 03:11:09.099212 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.099081    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xbn5d" podStartSLOduration=81.586245182 podStartE2EDuration="1m26.099064425s" totalImagesPullingTime="4.512819243s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:10:54.549450165 +0000 UTC m=+71.847590654" watchObservedRunningTime="2026-08-04 03:11:09.099064425 +0000 UTC m=+86.397204914"
Aug 04 03:11:09.099826 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.099809    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"]
Aug 04 03:11:09.102183 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.102169    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:09.105304 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.105281    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Aug 04 03:11:09.105432 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.105304    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Aug 04 03:11:09.105432 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.105334    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Aug 04 03:11:09.105595 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.105580    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Aug 04 03:11:09.110185 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.110166    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"]
Aug 04 03:11:09.151080 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.151048    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"]
Aug 04 03:11:09.153156 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.153140    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.156751 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.156730    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Aug 04 03:11:09.157007 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.156989    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Aug 04 03:11:09.157485 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.157464    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Aug 04 03:11:09.158009 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.157994    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Aug 04 03:11:09.184090 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.184065    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"]
Aug 04 03:11:09.206437 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.206405    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v429\" (UniqueName: \"kubernetes.io/projected/164d28bc-3f16-43a6-92da-e68ff8238a71-kube-api-access-2v429\") pod \"klusterlet-addon-workmgr-76494574b7-hndbn\" (UID: \"164d28bc-3f16-43a6-92da-e68ff8238a71\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:09.206437 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.206441    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-hub\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.206656 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.206457    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.206656 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.206475    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.206656 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.206495    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/164d28bc-3f16-43a6-92da-e68ff8238a71-tmp\") pod \"klusterlet-addon-workmgr-76494574b7-hndbn\" (UID: \"164d28bc-3f16-43a6-92da-e68ff8238a71\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:09.206656 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.206594    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-ca\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.206656 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.206616    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/164d28bc-3f16-43a6-92da-e68ff8238a71-klusterlet-config\") pod \"klusterlet-addon-workmgr-76494574b7-hndbn\" (UID: \"164d28bc-3f16-43a6-92da-e68ff8238a71\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:09.206656 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.206637    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzgdc\" (UniqueName: \"kubernetes.io/projected/199dd93a-8fa0-462b-92c1-7da419eb812a-kube-api-access-fzgdc\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.206656 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.206658    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/199dd93a-8fa0-462b-92c1-7da419eb812a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.307072 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.307024    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/164d28bc-3f16-43a6-92da-e68ff8238a71-tmp\") pod \"klusterlet-addon-workmgr-76494574b7-hndbn\" (UID: \"164d28bc-3f16-43a6-92da-e68ff8238a71\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:09.307072 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.307085    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-ca\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.307324 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.307102    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/164d28bc-3f16-43a6-92da-e68ff8238a71-klusterlet-config\") pod \"klusterlet-addon-workmgr-76494574b7-hndbn\" (UID: \"164d28bc-3f16-43a6-92da-e68ff8238a71\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:09.307324 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.307120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzgdc\" (UniqueName: \"kubernetes.io/projected/199dd93a-8fa0-462b-92c1-7da419eb812a-kube-api-access-fzgdc\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.307324 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.307140    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/199dd93a-8fa0-462b-92c1-7da419eb812a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.307324 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.307179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v429\" (UniqueName: \"kubernetes.io/projected/164d28bc-3f16-43a6-92da-e68ff8238a71-kube-api-access-2v429\") pod \"klusterlet-addon-workmgr-76494574b7-hndbn\" (UID: \"164d28bc-3f16-43a6-92da-e68ff8238a71\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:09.307324 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.307202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-hub\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.307590 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.307493    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/164d28bc-3f16-43a6-92da-e68ff8238a71-tmp\") pod \"klusterlet-addon-workmgr-76494574b7-hndbn\" (UID: \"164d28bc-3f16-43a6-92da-e68ff8238a71\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:09.307689 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.307664    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.307742 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.307709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.307938 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.307916    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/199dd93a-8fa0-462b-92c1-7da419eb812a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.309847 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.309820    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/164d28bc-3f16-43a6-92da-e68ff8238a71-klusterlet-config\") pod \"klusterlet-addon-workmgr-76494574b7-hndbn\" (UID: \"164d28bc-3f16-43a6-92da-e68ff8238a71\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:09.310012 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.309993    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.310107 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.310088    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-ca\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.310181 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.310159    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.310278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.310265    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/199dd93a-8fa0-462b-92c1-7da419eb812a-hub\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.315974 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.315949    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v429\" (UniqueName: \"kubernetes.io/projected/164d28bc-3f16-43a6-92da-e68ff8238a71-kube-api-access-2v429\") pod \"klusterlet-addon-workmgr-76494574b7-hndbn\" (UID: \"164d28bc-3f16-43a6-92da-e68ff8238a71\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:09.316064 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.316040    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzgdc\" (UniqueName: \"kubernetes.io/projected/199dd93a-8fa0-462b-92c1-7da419eb812a-kube-api-access-fzgdc\") pod \"cluster-proxy-proxy-agent-cffddc6b9-lr5m9\" (UID: \"199dd93a-8fa0-462b-92c1-7da419eb812a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.411476 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.411391    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:09.478221 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.478194    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:11:09.560121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.560093    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"]
Aug 04 03:11:09.562883 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:11:09.562851    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164d28bc_3f16_43a6_92da_e68ff8238a71.slice/crio-4b4027fd275e9ea5767caf866ad6de6773685bbb5c7e3667a9532512b977cfad WatchSource:0}: Error finding container 4b4027fd275e9ea5767caf866ad6de6773685bbb5c7e3667a9532512b977cfad: Status 404 returned error can't find the container with id 4b4027fd275e9ea5767caf866ad6de6773685bbb5c7e3667a9532512b977cfad
Aug 04 03:11:09.610284 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:09.610158    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"]
Aug 04 03:11:09.612732 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:11:09.612704    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod199dd93a_8fa0_462b_92c1_7da419eb812a.slice/crio-d6e9cd5b951ae93a6e4182705a22ad1936d9c602b9f4348cc446664cfcd8f9f1 WatchSource:0}: Error finding container d6e9cd5b951ae93a6e4182705a22ad1936d9c602b9f4348cc446664cfcd8f9f1: Status 404 returned error can't find the container with id d6e9cd5b951ae93a6e4182705a22ad1936d9c602b9f4348cc446664cfcd8f9f1
Aug 04 03:11:10.462424 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:10.462382    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:11:10.532704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:10.532634    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:11:10.560749 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:10.560652    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn" event={"ID":"164d28bc-3f16-43a6-92da-e68ff8238a71","Type":"ContainerStarted","Data":"4b4027fd275e9ea5767caf866ad6de6773685bbb5c7e3667a9532512b977cfad"}
Aug 04 03:11:10.563353 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:10.563272    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" event={"ID":"199dd93a-8fa0-462b-92c1-7da419eb812a","Type":"ContainerStarted","Data":"d6e9cd5b951ae93a6e4182705a22ad1936d9c602b9f4348cc446664cfcd8f9f1"}
Aug 04 03:11:15.580667 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:15.580630    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn" event={"ID":"164d28bc-3f16-43a6-92da-e68ff8238a71","Type":"ContainerStarted","Data":"f2fb18674ff5957a5ef935ed1def464710234fdbced16408a100c5c83017c532"}
Aug 04 03:11:15.581197 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:15.580829    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:15.582741 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:15.582718    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:11:15.583013 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:15.582992    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" event={"ID":"199dd93a-8fa0-462b-92c1-7da419eb812a","Type":"ContainerStarted","Data":"4727581fa87cfe02fa2c439657f21d357a910f5b2970105ca3f002ee5bee9b67"}
Aug 04 03:11:15.583087 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:15.583024    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" event={"ID":"199dd93a-8fa0-462b-92c1-7da419eb812a","Type":"ContainerStarted","Data":"bd365821619e04d061d03a2784e467ab2b30e23c9fb5279c9b45a8b7325c4044"}
Aug 04 03:11:15.583087 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:15.583035    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" event={"ID":"199dd93a-8fa0-462b-92c1-7da419eb812a","Type":"ContainerStarted","Data":"c2d8a3469ca3e6a1f82ac85ce0495d3810a0bc41cf78a683c44a5f9c246b0b72"}
Aug 04 03:11:15.595654 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:15.595610    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn" podStartSLOduration=0.824986298 podStartE2EDuration="6.595595334s" totalImagesPullingTime="5.770609036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:11:15.595004789 +0000 UTC m=+92.893145276" watchObservedRunningTime="2026-08-04 03:11:15.595595334 +0000 UTC m=+92.893735821"
Aug 04 03:11:15.627224 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:15.627174    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" podStartSLOduration=0.908730137 podStartE2EDuration="6.627157841s" totalImagesPullingTime="5.718427704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:11:15.626009069 +0000 UTC m=+92.924149558" watchObservedRunningTime="2026-08-04 03:11:15.627157841 +0000 UTC m=+92.925298329"
Aug 04 03:11:21.506326 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:21.506275    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:11:21.506737 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:11:21.506421    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 03:11:21.506737 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:11:21.506487    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls podName:a98e852a-83ee-40a7-8cc0-7243fcf6bafb nodeName:}" failed. No retries permitted until 2026-08-04 03:12:25.506469561 +0000 UTC m=+162.804610028 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls") pod "dns-default-78fkp" (UID: "a98e852a-83ee-40a7-8cc0-7243fcf6bafb") : secret "dns-default-metrics-tls" not found
Aug 04 03:11:25.520490 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:25.520453    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6lt8h"
Aug 04 03:11:53.026986 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:11:53.026931    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:11:53.027486 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:11:53.027079    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 04 03:11:53.027486 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:11:53.027152    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs podName:57750f19-0d04-432e-94f1-d0d0fde883bf nodeName:}" failed. No retries permitted until 2026-08-04 03:13:55.027135153 +0000 UTC m=+252.325275618 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs") pod "network-metrics-daemon-xlcd8" (UID: "57750f19-0d04-432e-94f1-d0d0fde883bf") : secret "metrics-daemon-secret" not found
Aug 04 03:12:20.585602 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:12:20.585559    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-78fkp" podUID="a98e852a-83ee-40a7-8cc0-7243fcf6bafb"
Aug 04 03:12:20.762067 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:12:20.762037    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-78fkp"
Aug 04 03:12:22.245333 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:12:22.245280    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:12:25.554087 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:12:25.554052    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:12:25.554648 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:12:25.554197    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 04 03:12:25.554648 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:12:25.554254    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls podName:a98e852a-83ee-40a7-8cc0-7243fcf6bafb nodeName:}" failed. No retries permitted until 2026-08-04 03:14:27.554239743 +0000 UTC m=+284.852380208 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls") pod "dns-default-78fkp" (UID: "a98e852a-83ee-40a7-8cc0-7243fcf6bafb") : secret "dns-default-metrics-tls" not found
Aug 04 03:12:35.232398 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:12:35.232321    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:13:15.581668 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:15.581614    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn" podUID="164d28bc-3f16-43a6-92da-e68ff8238a71" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused"
Aug 04 03:13:15.912491 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:15.912460    2559 generic.go:356] "Generic (PLEG): container finished" podID="164d28bc-3f16-43a6-92da-e68ff8238a71" containerID="f2fb18674ff5957a5ef935ed1def464710234fdbced16408a100c5c83017c532" exitCode=1
Aug 04 03:13:15.912675 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:15.912521    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn" event={"ID":"164d28bc-3f16-43a6-92da-e68ff8238a71","Type":"ContainerDied","Data":"f2fb18674ff5957a5ef935ed1def464710234fdbced16408a100c5c83017c532"}
Aug 04 03:13:15.912870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:15.912857    2559 scope.go:117] "RemoveContainer" containerID="f2fb18674ff5957a5ef935ed1def464710234fdbced16408a100c5c83017c532"
Aug 04 03:13:16.918721 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:16.918687    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn" event={"ID":"164d28bc-3f16-43a6-92da-e68ff8238a71","Type":"ContainerStarted","Data":"321cba5a20a078618cc01bd5d5158985ffe53ee58b8cc2a183b9628cb3343267"}
Aug 04 03:13:16.919143 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:16.918972    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:13:16.919615 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:16.919595    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76494574b7-hndbn"
Aug 04 03:13:24.384451 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:24.384413    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:51.409059 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:51.409023    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:51.504659 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:51.504627    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:53.629479 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:53.629444    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:54.731960 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:54.731928    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:54.834784 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:54.834724    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:55.056281 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.056249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:13:55.056448 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:55.056352    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Aug 04 03:13:55.056448 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:55.056410    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs podName:57750f19-0d04-432e-94f1-d0d0fde883bf nodeName:}" failed. No retries permitted until 2026-08-04 03:15:57.0563965 +0000 UTC m=+374.354536966 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs") pod "network-metrics-daemon-xlcd8" (UID: "57750f19-0d04-432e-94f1-d0d0fde883bf") : secret "metrics-daemon-secret" not found
Aug 04 03:13:55.291138 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.291097    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:55.341141 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.341057    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:55.612212 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.612139    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:55.713088 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.713055    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:55.973133 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.973057    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k8plp"]
Aug 04 03:13:55.975103 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.975002    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:55.977060 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.977033    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 04 03:13:55.977201 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.977058    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 04 03:13:55.977201 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.977091    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2m679\""
Aug 04 03:13:55.977849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.977835    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 04 03:13:55.977906 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.977857    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 04 03:13:55.988259 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.988234    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k8plp"]
Aug 04 03:13:55.994688 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:55.994656    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:56.061159 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.061125    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-crio-socket\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.061344 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.061185    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.061344 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.061239    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlcbz\" (UniqueName: \"kubernetes.io/projected/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-api-access-hlcbz\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.061344 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.061276    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-data-volume\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.061344 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.061293    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.070398 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.070373    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:56.161926 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.161893    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-crio-socket\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.162081 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.161958    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.162081 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.161989    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlcbz\" (UniqueName: \"kubernetes.io/projected/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-api-access-hlcbz\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.162081 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.162017    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-crio-socket\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.162081 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.162020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-data-volume\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.162081 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.162067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.162318 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:56.162142    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 03:13:56.162318 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:56.162176    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 03:13:56.162318 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:56.162208    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm podName:6da53b56-3fef-43d6-8a4b-7d7b590d0aa3 nodeName:}" failed. No retries permitted until 2026-08-04 03:13:56.662192186 +0000 UTC m=+253.960332655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k8plp" (UID: "6da53b56-3fef-43d6-8a4b-7d7b590d0aa3") : configmap "kube-rbac-proxy" not found
Aug 04 03:13:56.162318 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:56.162228    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls podName:6da53b56-3fef-43d6-8a4b-7d7b590d0aa3 nodeName:}" failed. No retries permitted until 2026-08-04 03:13:56.662215729 +0000 UTC m=+253.960356195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k8plp" (UID: "6da53b56-3fef-43d6-8a4b-7d7b590d0aa3") : secret "insights-runtime-extractor-tls" not found
Aug 04 03:13:56.162318 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.162313    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-data-volume\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.170012 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.169989    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlcbz\" (UniqueName: \"kubernetes.io/projected/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-api-access-hlcbz\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.232245 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.232156    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:56.289686 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.289652    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:56.664090 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.663987    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.664090 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:56.664045    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:56.664316 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:56.664109    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 03:13:56.664316 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:56.664155    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm podName:6da53b56-3fef-43d6-8a4b-7d7b590d0aa3 nodeName:}" failed. No retries permitted until 2026-08-04 03:13:57.664141232 +0000 UTC m=+254.962281698 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k8plp" (UID: "6da53b56-3fef-43d6-8a4b-7d7b590d0aa3") : configmap "kube-rbac-proxy" not found
Aug 04 03:13:56.664316 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:56.664159    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 03:13:56.664316 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:56.664223    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls podName:6da53b56-3fef-43d6-8a4b-7d7b590d0aa3 nodeName:}" failed. No retries permitted until 2026-08-04 03:13:57.664205262 +0000 UTC m=+254.962345733 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k8plp" (UID: "6da53b56-3fef-43d6-8a4b-7d7b590d0aa3") : secret "insights-runtime-extractor-tls" not found
Aug 04 03:13:57.181072 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:57.181038    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:57.234737 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:57.234699    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:57.674829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:57.674789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:57.674829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:57.674834    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:57.675040 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:57.674916    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 03:13:57.675040 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:57.674934    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 03:13:57.675040 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:57.674969    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm podName:6da53b56-3fef-43d6-8a4b-7d7b590d0aa3 nodeName:}" failed. No retries permitted until 2026-08-04 03:13:59.674955149 +0000 UTC m=+256.973095615 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k8plp" (UID: "6da53b56-3fef-43d6-8a4b-7d7b590d0aa3") : configmap "kube-rbac-proxy" not found
Aug 04 03:13:57.675040 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:57.674994    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls podName:6da53b56-3fef-43d6-8a4b-7d7b590d0aa3 nodeName:}" failed. No retries permitted until 2026-08-04 03:13:59.674981656 +0000 UTC m=+256.973122122 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k8plp" (UID: "6da53b56-3fef-43d6-8a4b-7d7b590d0aa3") : secret "insights-runtime-extractor-tls" not found
Aug 04 03:13:57.943327 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:57.943264    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f4mrm_6864c31c-d3e5-4224-beda-e85b0d20868d/dns-node-resolver/0.log"
Aug 04 03:13:58.024128 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:58.024096    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:58.742450 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:58.742426    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9m9ks_7a2ea3c8-88ca-4802-987b-be5942899884/node-ca/0.log"
Aug 04 03:13:59.068007 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:59.067914    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:59.231757 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:59.231728    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:59.332050 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:59.331963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:59.434644 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:59.434609    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:59.539705 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:59.539670    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:13:59.690771 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:59.690676    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:59.690771 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:59.690737    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:13:59.690957 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:59.690812    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 03:13:59.690957 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:59.690833    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 03:13:59.690957 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:59.690885    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm podName:6da53b56-3fef-43d6-8a4b-7d7b590d0aa3 nodeName:}" failed. No retries permitted until 2026-08-04 03:14:03.690869072 +0000 UTC m=+260.989009540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k8plp" (UID: "6da53b56-3fef-43d6-8a4b-7d7b590d0aa3") : configmap "kube-rbac-proxy" not found
Aug 04 03:13:59.690957 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:13:59.690899    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls podName:6da53b56-3fef-43d6-8a4b-7d7b590d0aa3 nodeName:}" failed. No retries permitted until 2026-08-04 03:14:03.690893806 +0000 UTC m=+260.989034272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k8plp" (UID: "6da53b56-3fef-43d6-8a4b-7d7b590d0aa3") : secret "insights-runtime-extractor-tls" not found
Aug 04 03:13:59.742817 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:59.742792    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xqt4c_5eb1ab5d-b609-4822-baba-4357e434097f/serve-healthcheck-canary/0.log"
Aug 04 03:13:59.761401 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:13:59.761373    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:00.564630 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:00.564597    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:00.817990 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:00.817907    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:01.162611 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:01.162522    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:01.371714 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:01.371682    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:01.569423 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:01.569397    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:01.961602 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:01.961517    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:02.358048 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:02.358016    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:02.962214 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:02.962182    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:03.024307 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:03.024274    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:03.360155 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:03.360126    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:03.561224 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:03.561194    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:03.719846 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:03.719754    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:14:03.719846 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:03.719809    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:14:03.720023 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:14:03.719896    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 04 03:14:03.720023 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:14:03.719910    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 04 03:14:03.720023 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:14:03.719957    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm podName:6da53b56-3fef-43d6-8a4b-7d7b590d0aa3 nodeName:}" failed. No retries permitted until 2026-08-04 03:14:11.719942011 +0000 UTC m=+269.018082482 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k8plp" (UID: "6da53b56-3fef-43d6-8a4b-7d7b590d0aa3") : configmap "kube-rbac-proxy" not found
Aug 04 03:14:03.720023 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:14:03.719971    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls podName:6da53b56-3fef-43d6-8a4b-7d7b590d0aa3 nodeName:}" failed. No retries permitted until 2026-08-04 03:14:11.719965506 +0000 UTC m=+269.018105972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k8plp" (UID: "6da53b56-3fef-43d6-8a4b-7d7b590d0aa3") : secret "insights-runtime-extractor-tls" not found
Aug 04 03:14:03.770145 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:03.770114    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:04.066991 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:04.066961    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:04.167125 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:04.167086    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:04.360175 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:04.360081    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:04.762733 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:04.762704    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:05.370054 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:05.370026    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:05.558327 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:05.558293    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:05.763575 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:05.763537    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:06.157968 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:06.157885    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:06.560990 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:06.560949    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:06.761734 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:06.761696    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:07.158509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:07.158475    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:07.960512 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:07.960478    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:08.360368 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:08.360325    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:08.509286 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:08.509239    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:08.610442 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:08.610359    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:08.759261 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:08.759223    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:08.965128 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:08.965046    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:09.561162 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:09.561132    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:09.788723 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:09.788691    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:09.888357 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:09.888262    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:09.988245 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:09.988214    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:10.219368 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:10.219265    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:11.778712 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:11.778677    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:14:11.778712 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:11.778720    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:14:11.779414 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:11.779395    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:14:11.781007 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:11.780992    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6da53b56-3fef-43d6-8a4b-7d7b590d0aa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k8plp\" (UID: \"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3\") " pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:14:11.884112 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:11.884074    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k8plp"
Aug 04 03:14:12.003670 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:12.003643    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k8plp"]
Aug 04 03:14:12.006768 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:14:12.006745    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da53b56_3fef_43d6_8a4b_7d7b590d0aa3.slice/crio-f22caf0daa5c1b55b7b5e95e069dc43211d2d7a57f71507af2226ff8e03f0f6d WatchSource:0}: Error finding container f22caf0daa5c1b55b7b5e95e069dc43211d2d7a57f71507af2226ff8e03f0f6d: Status 404 returned error can't find the container with id f22caf0daa5c1b55b7b5e95e069dc43211d2d7a57f71507af2226ff8e03f0f6d
Aug 04 03:14:12.075111 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:12.075079    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k8plp" event={"ID":"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3","Type":"ContainerStarted","Data":"d7e4c6e8f9fe60b299fcb3eab09a531648d98f07ae382f97323c005c68ec0b6c"}
Aug 04 03:14:12.075303 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:12.075161    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k8plp" event={"ID":"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3","Type":"ContainerStarted","Data":"f22caf0daa5c1b55b7b5e95e069dc43211d2d7a57f71507af2226ff8e03f0f6d"}
Aug 04 03:14:13.079162 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:13.079124    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k8plp" event={"ID":"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3","Type":"ContainerStarted","Data":"7ad38cd7d0d901b703e7f57e3e925a170fbf448817e71497439b3dc8a9539b62"}
Aug 04 03:14:14.084314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:14.084280    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k8plp" event={"ID":"6da53b56-3fef-43d6-8a4b-7d7b590d0aa3","Type":"ContainerStarted","Data":"f58898e72cf154ec513402c7b4bb734b0341c4a3e070757d9022b016df7b474a"}
Aug 04 03:14:14.100839 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:14.100791    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k8plp" podStartSLOduration=17.230325704 podStartE2EDuration="19.100776083s" totalImagesPullingTime="1.870450379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 03:13:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:14:14.100241721 +0000 UTC m=+271.398382210" watchObservedRunningTime="2026-08-04 03:14:14.100776083 +0000 UTC m=+271.398916571"
Aug 04 03:14:14.961435 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:14.961389    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:15.062315 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:15.062283    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:15.116819 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:15.116784    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:15.257855 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:15.257815    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:15.435147 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:15.435121    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:15.632721 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:15.632635    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:15.733104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:15.733072    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:16.060125 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:16.060093    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:16.115053 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:16.115020    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:16.262209 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:16.262170    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:16.321133 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:16.321056    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:16.377984 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:16.377955    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:16.478831 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:16.478799    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:16.641473 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:16.641394    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:16.843528 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:16.843495    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:17.698913 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:17.698875    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:18.659479 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:18.659441    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:18.712639 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:18.712603    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:19.770646 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.770602    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"]
Aug 04 03:14:19.772707 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.772691    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.776787 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.776766    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 04 03:14:19.778870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.778848    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 04 03:14:19.778969 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.778932    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9lvw5\""
Aug 04 03:14:19.779582 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.779568    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 04 03:14:19.807265 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.807241    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 04 03:14:19.820477 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.820450    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"]
Aug 04 03:14:19.840330 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.840301    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z6ms\" (UniqueName: \"kubernetes.io/projected/977d3bf8-d024-4c65-85f6-5399b084433a-kube-api-access-4z6ms\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.840475 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.840335    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/977d3bf8-d024-4c65-85f6-5399b084433a-registry-tls\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.840475 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.840402    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/977d3bf8-d024-4c65-85f6-5399b084433a-bound-sa-token\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.840475 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.840439    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/977d3bf8-d024-4c65-85f6-5399b084433a-ca-trust-extracted\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.840647 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.840480    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/977d3bf8-d024-4c65-85f6-5399b084433a-image-registry-private-configuration\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.840647 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.840555    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/977d3bf8-d024-4c65-85f6-5399b084433a-trusted-ca\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.840647 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.840610    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/977d3bf8-d024-4c65-85f6-5399b084433a-registry-certificates\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.840771 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.840650    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/977d3bf8-d024-4c65-85f6-5399b084433a-installation-pull-secrets\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.878326 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.878292    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:19.941320 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.941289    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/977d3bf8-d024-4c65-85f6-5399b084433a-ca-trust-extracted\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.941442 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.941343    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/977d3bf8-d024-4c65-85f6-5399b084433a-image-registry-private-configuration\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.941442 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.941372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/977d3bf8-d024-4c65-85f6-5399b084433a-trusted-ca\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.941442 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.941398    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/977d3bf8-d024-4c65-85f6-5399b084433a-registry-certificates\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.941442 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.941415    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/977d3bf8-d024-4c65-85f6-5399b084433a-installation-pull-secrets\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.941442 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.941438    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z6ms\" (UniqueName: \"kubernetes.io/projected/977d3bf8-d024-4c65-85f6-5399b084433a-kube-api-access-4z6ms\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.941719 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.941662    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/977d3bf8-d024-4c65-85f6-5399b084433a-registry-tls\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.941719 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.941713    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/977d3bf8-d024-4c65-85f6-5399b084433a-bound-sa-token\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.941820 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.941756    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/977d3bf8-d024-4c65-85f6-5399b084433a-ca-trust-extracted\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.942126 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.942098    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/977d3bf8-d024-4c65-85f6-5399b084433a-registry-certificates\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.942438 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.942420    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/977d3bf8-d024-4c65-85f6-5399b084433a-trusted-ca\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.944184 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.944163    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/977d3bf8-d024-4c65-85f6-5399b084433a-image-registry-private-configuration\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.944644 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.944625    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/977d3bf8-d024-4c65-85f6-5399b084433a-installation-pull-secrets\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.944692 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.944625    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/977d3bf8-d024-4c65-85f6-5399b084433a-registry-tls\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.949494 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.949472    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/977d3bf8-d024-4c65-85f6-5399b084433a-bound-sa-token\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.949913 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.949896    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z6ms\" (UniqueName: \"kubernetes.io/projected/977d3bf8-d024-4c65-85f6-5399b084433a-kube-api-access-4z6ms\") pod \"image-registry-5dbf7b5fcf-j7pqz\" (UID: \"977d3bf8-d024-4c65-85f6-5399b084433a\") " pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:19.976942 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:19.976899    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:20.077905 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:20.077817    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:20.080789 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:20.080774    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:20.205377 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:20.205345    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"]
Aug 04 03:14:20.208557 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:14:20.208522    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod977d3bf8_d024_4c65_85f6_5399b084433a.slice/crio-c13ff614a00e6e6c22ffde9dd5f0a49388d63a2a74c547b4db08f9df6db85e3c WatchSource:0}: Error finding container c13ff614a00e6e6c22ffde9dd5f0a49388d63a2a74c547b4db08f9df6db85e3c: Status 404 returned error can't find the container with id c13ff614a00e6e6c22ffde9dd5f0a49388d63a2a74c547b4db08f9df6db85e3c
Aug 04 03:14:21.103935 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:21.103901    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz" event={"ID":"977d3bf8-d024-4c65-85f6-5399b084433a","Type":"ContainerStarted","Data":"f055064d45f4625718ec66ce331c05a600b61075477013cb997d6f078a29bbf7"}
Aug 04 03:14:21.104426 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:21.103954    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz" event={"ID":"977d3bf8-d024-4c65-85f6-5399b084433a","Type":"ContainerStarted","Data":"c13ff614a00e6e6c22ffde9dd5f0a49388d63a2a74c547b4db08f9df6db85e3c"}
Aug 04 03:14:21.104426 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:21.104022    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:21.121375 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:21.121328    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz" podStartSLOduration=2.121312301 podStartE2EDuration="2.121312301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:14:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:14:21.120968742 +0000 UTC m=+278.419109251" watchObservedRunningTime="2026-08-04 03:14:21.121312301 +0000 UTC m=+278.419452788"
Aug 04 03:14:22.004298 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:22.004263    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:22.065404 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:22.065369    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:22.220195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:22.220159    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:23.287399 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:23.287371    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:23.762715 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:14:23.762673    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-78fkp" podUID="a98e852a-83ee-40a7-8cc0-7243fcf6bafb"
Aug 04 03:14:24.112657 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:24.112575    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-78fkp"
Aug 04 03:14:24.435751 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:24.435668    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:27.122957 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.122930    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:27.329201 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.329164    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:27.395976 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.395896    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-995qc"]
Aug 04 03:14:27.399076 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.399060    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.401145 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.401125    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 04 03:14:27.401398 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.401382    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 04 03:14:27.401522 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.401496    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 04 03:14:27.401769 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.401754    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 04 03:14:27.402513 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.402494    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-h8mnb\""
Aug 04 03:14:27.402612 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.402594    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 04 03:14:27.402686 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.402596    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 04 03:14:27.422499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.422351    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:27.487560 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.487516    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:27.498533 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.498499    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-wtmp\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.498685 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.498569    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f859f8e-2f0f-4072-8886-cf427010281b-metrics-client-ca\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.498685 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.498608    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.498685 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.498660    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-accelerators-collector-config\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.498841 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.498702    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7st46\" (UniqueName: \"kubernetes.io/projected/1f859f8e-2f0f-4072-8886-cf427010281b-kube-api-access-7st46\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.498841 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.498727    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f859f8e-2f0f-4072-8886-cf427010281b-sys\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.498841 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.498754    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f859f8e-2f0f-4072-8886-cf427010281b-root\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.498841 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.498776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-tls\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.498841 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.498828    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-textfile\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.586272 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.586230    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:27.599153 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599127    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.599307 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599162    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:14:27.599307 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599187    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-accelerators-collector-config\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.599307 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599231    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7st46\" (UniqueName: \"kubernetes.io/projected/1f859f8e-2f0f-4072-8886-cf427010281b-kube-api-access-7st46\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.599307 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599257    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f859f8e-2f0f-4072-8886-cf427010281b-sys\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.599307 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599282    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f859f8e-2f0f-4072-8886-cf427010281b-root\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.599307 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599302    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-tls\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.599639 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599344    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-textfile\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.599639 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599374    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-wtmp\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.599639 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599398    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f859f8e-2f0f-4072-8886-cf427010281b-sys\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.599639 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:14:27.599497    2559 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Aug 04 03:14:27.599639 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:14:27.599584    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-tls podName:1f859f8e-2f0f-4072-8886-cf427010281b nodeName:}" failed. No retries permitted until 2026-08-04 03:14:28.099563658 +0000 UTC m=+285.397704124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-tls") pod "node-exporter-995qc" (UID: "1f859f8e-2f0f-4072-8886-cf427010281b") : secret "node-exporter-tls" not found
Aug 04 03:14:27.599900 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599846    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-textfile\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.599900 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599410    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f859f8e-2f0f-4072-8886-cf427010281b-metrics-client-ca\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.599900 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599893    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-wtmp\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.600040 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599892    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-accelerators-collector-config\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.600040 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599922    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f859f8e-2f0f-4072-8886-cf427010281b-metrics-client-ca\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.600040 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.599959    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f859f8e-2f0f-4072-8886-cf427010281b-root\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.601502 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.601483    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a98e852a-83ee-40a7-8cc0-7243fcf6bafb-metrics-tls\") pod \"dns-default-78fkp\" (UID: \"a98e852a-83ee-40a7-8cc0-7243fcf6bafb\") " pod="openshift-dns/dns-default-78fkp"
Aug 04 03:14:27.601502 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.601496    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.608403 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.608382    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7st46\" (UniqueName: \"kubernetes.io/projected/1f859f8e-2f0f-4072-8886-cf427010281b-kube-api-access-7st46\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:27.679437 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.679349    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:27.715963 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.715936    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4d7vw\""
Aug 04 03:14:27.724827 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.724802    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-78fkp"
Aug 04 03:14:27.851486 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:27.851461    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-78fkp"]
Aug 04 03:14:27.853809 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:14:27.853784    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98e852a_83ee_40a7_8cc0_7243fcf6bafb.slice/crio-b1eda5ad427e34f890ca28dbdb7f89507b8b667bd0e83784c155bb456772f9b2 WatchSource:0}: Error finding container b1eda5ad427e34f890ca28dbdb7f89507b8b667bd0e83784c155bb456772f9b2: Status 404 returned error can't find the container with id b1eda5ad427e34f890ca28dbdb7f89507b8b667bd0e83784c155bb456772f9b2
Aug 04 03:14:28.068477 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:28.068447    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:28.102558 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:28.102522    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-tls\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:28.104790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:28.104768    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f859f8e-2f0f-4072-8886-cf427010281b-node-exporter-tls\") pod \"node-exporter-995qc\" (UID: \"1f859f8e-2f0f-4072-8886-cf427010281b\") " pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:28.124950 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:28.124917    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-78fkp" event={"ID":"a98e852a-83ee-40a7-8cc0-7243fcf6bafb","Type":"ContainerStarted","Data":"b1eda5ad427e34f890ca28dbdb7f89507b8b667bd0e83784c155bb456772f9b2"}
Aug 04 03:14:28.308507 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:28.308466    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-995qc"
Aug 04 03:14:28.318402 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:14:28.318375    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f859f8e_2f0f_4072_8886_cf427010281b.slice/crio-106cb6020dcc0b5ee5365ac4861ad4cc2ef2f62d032f7dd78e5c20c88257801f WatchSource:0}: Error finding container 106cb6020dcc0b5ee5365ac4861ad4cc2ef2f62d032f7dd78e5c20c88257801f: Status 404 returned error can't find the container with id 106cb6020dcc0b5ee5365ac4861ad4cc2ef2f62d032f7dd78e5c20c88257801f
Aug 04 03:14:28.340504 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:28.340468    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:28.425536 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:28.425502    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:28.540958 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:28.540921    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:28.643933 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:28.643838    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:29.129893 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:29.129860    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-995qc" event={"ID":"1f859f8e-2f0f-4072-8886-cf427010281b","Type":"ContainerStarted","Data":"106cb6020dcc0b5ee5365ac4861ad4cc2ef2f62d032f7dd78e5c20c88257801f"}
Aug 04 03:14:30.134492 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:30.134456    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-78fkp" event={"ID":"a98e852a-83ee-40a7-8cc0-7243fcf6bafb","Type":"ContainerStarted","Data":"94c6bbbf6eaeac3b90730c23327948b2380231220f825be0b2bc47ad97b25b81"}
Aug 04 03:14:30.134978 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:30.134505    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-78fkp" event={"ID":"a98e852a-83ee-40a7-8cc0-7243fcf6bafb","Type":"ContainerStarted","Data":"0f4808ac19fcb45b536b5f1dbceb21cbd478d467236efad9798468358cde2d50"}
Aug 04 03:14:30.134978 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:30.134566    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-78fkp"
Aug 04 03:14:30.136109 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:30.136088    2559 generic.go:356] "Generic (PLEG): container finished" podID="1f859f8e-2f0f-4072-8886-cf427010281b" containerID="091945db4a463ab27513218fd3c9acf444102813e5a308e5df9f7d9fdffea4e4" exitCode=0
Aug 04 03:14:30.136193 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:30.136132    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-995qc" event={"ID":"1f859f8e-2f0f-4072-8886-cf427010281b","Type":"ContainerDied","Data":"091945db4a463ab27513218fd3c9acf444102813e5a308e5df9f7d9fdffea4e4"}
Aug 04 03:14:30.153860 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:30.153819    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-78fkp" podStartSLOduration=251.554577787 podStartE2EDuration="4m13.153802537s" totalImagesPullingTime="1.59922475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:14:30.152652823 +0000 UTC m=+287.450793310" watchObservedRunningTime="2026-08-04 03:14:30.153802537 +0000 UTC m=+287.451943024"
Aug 04 03:14:30.342499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:30.342469    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:30.398781 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:30.398692    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:30.484158 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:30.484128    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:31.081995 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:31.081940    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:31.142048 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:31.142014    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-995qc" event={"ID":"1f859f8e-2f0f-4072-8886-cf427010281b","Type":"ContainerStarted","Data":"1cd66c45ca4d0b3b2ed3a052222960894d2fa7b06368e45312fdbfb8e3e4bd66"}
Aug 04 03:14:31.142430 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:31.142097    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-995qc" event={"ID":"1f859f8e-2f0f-4072-8886-cf427010281b","Type":"ContainerStarted","Data":"d48d87a8b5ee1da1bef6c3a1b209fde3c397156f88631bdb652b31d1d9113629"}
Aug 04 03:14:31.162634 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:31.162585    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-995qc" podStartSLOduration=2.954422605 podStartE2EDuration="4.162563815s" totalImagesPullingTime="1.133152698s" totalInitContainerRuntime="74.988512ms" isStatefulPod=true podCreationTimestamp="2026-08-04 03:14:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:14:31.161343231 +0000 UTC m=+288.459483718" watchObservedRunningTime="2026-08-04 03:14:31.162563815 +0000 UTC m=+288.460704300"
Aug 04 03:14:32.077719 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.077679    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:32.144577 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.144523    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:32.573761 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.573731    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9479f9886-nhhhl"]
Aug 04 03:14:32.576800 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.576783    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.579604 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.579582    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Aug 04 03:14:32.579727 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.579607    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Aug 04 03:14:32.579727 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.579639    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-d8jt9\""
Aug 04 03:14:32.579905 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.579890    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Aug 04 03:14:32.580845 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.580831    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Aug 04 03:14:32.580976 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.580961    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Aug 04 03:14:32.585227 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.585209    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Aug 04 03:14:32.605443 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.605418    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9479f9886-nhhhl"]
Aug 04 03:14:32.638744 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.638705    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.638744 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.638746    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-serving-certs-ca-bundle\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.638963 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.638767    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.638963 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.638839    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-metrics-client-ca\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.638963 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.638873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-telemeter-client-tls\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.638963 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.638953    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-federate-client-tls\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.639099 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.638986    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-secret-telemeter-client\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.639099 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.639014    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qxnc\" (UniqueName: \"kubernetes.io/projected/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-kube-api-access-6qxnc\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.666746 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.666711    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:32.739550 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.739516    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-secret-telemeter-client\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.739672 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.739590    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qxnc\" (UniqueName: \"kubernetes.io/projected/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-kube-api-access-6qxnc\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.739672 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.739631    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.739672 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.739663    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-serving-certs-ca-bundle\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.739818 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.739684    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.739818 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.739752    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-metrics-client-ca\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.739818 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.739802    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-telemeter-client-tls\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.739961 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.739887    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-federate-client-tls\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.740528 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.740497    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-serving-certs-ca-bundle\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.740664 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.740512    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-metrics-client-ca\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.740716 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.740703    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.742660 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.742636    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-federate-client-tls\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.742723 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.742691    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.742759 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.742728    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-telemeter-client-tls\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.742799 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.742773    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-secret-telemeter-client\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.747680 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.747654    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qxnc\" (UniqueName: \"kubernetes.io/projected/9decef81-74a7-4f6e-a2e6-befe2a57ef1e-kube-api-access-6qxnc\") pod \"telemeter-client-9479f9886-nhhhl\" (UID: \"9decef81-74a7-4f6e-a2e6-befe2a57ef1e\") " pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.885605 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.885499    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl"
Aug 04 03:14:32.887238 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.887212    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:32.942395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:32.942367    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:33.020117 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:33.020088    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9479f9886-nhhhl"]
Aug 04 03:14:33.023026 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:14:33.023001    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9decef81_74a7_4f6e_a2e6_befe2a57ef1e.slice/crio-e06430083790e8ab7b7ddadfbb9b5e8d7ea60e69e1c85a402f08c39fa90c1831 WatchSource:0}: Error finding container e06430083790e8ab7b7ddadfbb9b5e8d7ea60e69e1c85a402f08c39fa90c1831: Status 404 returned error can't find the container with id e06430083790e8ab7b7ddadfbb9b5e8d7ea60e69e1c85a402f08c39fa90c1831
Aug 04 03:14:33.154572 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:33.154465    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl" event={"ID":"9decef81-74a7-4f6e-a2e6-befe2a57ef1e","Type":"ContainerStarted","Data":"e06430083790e8ab7b7ddadfbb9b5e8d7ea60e69e1c85a402f08c39fa90c1831"}
Aug 04 03:14:33.545067 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:33.545028    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:35.162556 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:35.162513    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl" event={"ID":"9decef81-74a7-4f6e-a2e6-befe2a57ef1e","Type":"ContainerStarted","Data":"41caa19481c1951ebcad6edd7d898ac1902534da0e5198a874e0f86f83bbc72a"}
Aug 04 03:14:36.167803 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:36.167769    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl" event={"ID":"9decef81-74a7-4f6e-a2e6-befe2a57ef1e","Type":"ContainerStarted","Data":"ab796b4b04efefc6a76951795015f03cfc865db86a75a92069111b7bf648b47c"}
Aug 04 03:14:36.802393 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:36.802350    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:36.906314 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:36.906282    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:37.004625 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:37.004596    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:37.173227 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:37.173140    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl" event={"ID":"9decef81-74a7-4f6e-a2e6-befe2a57ef1e","Type":"ContainerStarted","Data":"fb86805e76e7e5a07ac907e2c0464b0fccaadf543324d18c4f6a07f51d97706a"}
Aug 04 03:14:37.194515 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:37.194455    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9479f9886-nhhhl" podStartSLOduration=2.213690353 podStartE2EDuration="5.194434277s" totalImagesPullingTime="2.980743924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:14:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:14:37.193469637 +0000 UTC m=+294.491610202" watchObservedRunningTime="2026-08-04 03:14:37.194434277 +0000 UTC m=+294.492574769"
Aug 04 03:14:38.232966 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:14:38.232923    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xlcd8" podUID="57750f19-0d04-432e-94f1-d0d0fde883bf"
Aug 04 03:14:38.637701 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:38.637610    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:38.732184 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:38.732153    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:39.109720 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:39.109676    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:39.210331 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:39.210295    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:40.144587 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:40.144561    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-78fkp"
Aug 04 03:14:42.000556 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:42.000500    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:42.107187 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:42.107154    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:42.110453 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:42.110430    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5dbf7b5fcf-j7pqz"
Aug 04 03:14:42.184733 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:42.184698    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:42.257593 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:42.257490    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:42.328814 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:42.328782    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:43.082717 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:43.082687    2559 kubelet.go:1648] "Image garbage collection succeeded"
Aug 04 03:14:45.781713 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:45.781680    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:45.833821 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:45.833787    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:46.378787 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:46.378754    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:46.478929 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:46.478897    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:47.270275 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:47.270241    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:47.321926 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:47.321893    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:48.268927 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:48.268893    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:48.320777 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:48.320749    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:50.232096 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:50.232057    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:14:50.990440 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:50.990401    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:51.046479 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:51.046438    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:51.188859 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:51.188828    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:51.289166 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:51.289076    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:54.796252 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:54.796207    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:54.896195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:54.896160    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:14:59.480503 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:14:59.480462    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" podUID="199dd93a-8fa0-462b-92c1-7da419eb812a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 04 03:15:01.378971 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:01.378933    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:15:01.581234 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:01.581197    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:15:09.480365 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:09.480317    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" podUID="199dd93a-8fa0-462b-92c1-7da419eb812a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 04 03:15:19.479561 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:19.479508    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" podUID="199dd93a-8fa0-462b-92c1-7da419eb812a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Aug 04 03:15:19.479916 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:19.479600    2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9"
Aug 04 03:15:19.480069 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:19.480032    2559 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"4727581fa87cfe02fa2c439657f21d357a910f5b2970105ca3f002ee5bee9b67"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Aug 04 03:15:19.480116 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:19.480079    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" podUID="199dd93a-8fa0-462b-92c1-7da419eb812a" containerName="service-proxy" containerID="cri-o://4727581fa87cfe02fa2c439657f21d357a910f5b2970105ca3f002ee5bee9b67" gracePeriod=30
Aug 04 03:15:20.323373 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:20.323336    2559 generic.go:356] "Generic (PLEG): container finished" podID="199dd93a-8fa0-462b-92c1-7da419eb812a" containerID="4727581fa87cfe02fa2c439657f21d357a910f5b2970105ca3f002ee5bee9b67" exitCode=2
Aug 04 03:15:20.323566 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:20.323393    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" event={"ID":"199dd93a-8fa0-462b-92c1-7da419eb812a","Type":"ContainerDied","Data":"4727581fa87cfe02fa2c439657f21d357a910f5b2970105ca3f002ee5bee9b67"}
Aug 04 03:15:20.323566 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:20.323437    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cffddc6b9-lr5m9" event={"ID":"199dd93a-8fa0-462b-92c1-7da419eb812a","Type":"ContainerStarted","Data":"ea1dc7047d541267b3904da4c4cbced54bf53718bc9848cb54adb87d8e496008"}
Aug 04 03:15:43.420466 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:43.420377    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:15:43.471846 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:43.471808    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:15:46.193217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:46.193165    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:15:46.250438 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:46.250406    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:15:53.853680 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:53.853640    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:15:53.910700 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:53.910656    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:15:55.219235 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:55.219196    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:15:55.273439 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:55.273399    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:15:57.120865 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:57.120823    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:15:57.123138 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:57.123110    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57750f19-0d04-432e-94f1-d0d0fde883bf-metrics-certs\") pod \"network-metrics-daemon-xlcd8\" (UID: \"57750f19-0d04-432e-94f1-d0d0fde883bf\") " pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:15:57.135409 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:57.135380    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6l47c\""
Aug 04 03:15:57.143184 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:57.143161    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlcd8"
Aug 04 03:15:57.266444 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:57.266421    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xlcd8"]
Aug 04 03:15:57.268891 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:15:57.268855    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57750f19_0d04_432e_94f1_d0d0fde883bf.slice/crio-8f31307f5acba425bbdc441797386a06433ee7e275b2837f17e321d5c2e6491e WatchSource:0}: Error finding container 8f31307f5acba425bbdc441797386a06433ee7e275b2837f17e321d5c2e6491e: Status 404 returned error can't find the container with id 8f31307f5acba425bbdc441797386a06433ee7e275b2837f17e321d5c2e6491e
Aug 04 03:15:57.270968 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:57.270952    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 03:15:57.440379 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:57.440296    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlcd8" event={"ID":"57750f19-0d04-432e-94f1-d0d0fde883bf","Type":"ContainerStarted","Data":"8f31307f5acba425bbdc441797386a06433ee7e275b2837f17e321d5c2e6491e"}
Aug 04 03:15:58.446023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:58.445943    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlcd8" event={"ID":"57750f19-0d04-432e-94f1-d0d0fde883bf","Type":"ContainerStarted","Data":"e8728e660c324fcb0deaf2d6e4bc2bcf017debde70f5e988403bc83c9d48a5d3"}
Aug 04 03:15:58.446023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:58.446020    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlcd8" event={"ID":"57750f19-0d04-432e-94f1-d0d0fde883bf","Type":"ContainerStarted","Data":"b73fe55ea25fca2d3ecf7e89c8f67d26b0db228241bfbd25460aef5fe3f78861"}
Aug 04 03:15:58.461331 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:58.461278    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xlcd8" podStartSLOduration=374.642946262 podStartE2EDuration="6m15.461262272s" totalImagesPullingTime="818.31601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:15:58.460623988 +0000 UTC m=+375.758764495" watchObservedRunningTime="2026-08-04 03:15:58.461262272 +0000 UTC m=+375.759402759"
Aug 04 03:15:58.552176 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:58.552135    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:15:58.623012 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:15:58.622976    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:16:01.396566 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:16:01.396514    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:16:01.491522 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:16:01.491483    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:16:08.397288 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:16:08.397245    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:16:08.498179 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:16:08.498144    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:16:19.413010 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:16:19.412967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:16:19.530922 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:16:19.530873    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:16:19.581643 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:16:19.581608    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:16:36.182293 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:16:36.182257    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:16:36.235331 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:16:36.235294    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:00.192430 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:00.192391    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:00.288030 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:00.287997    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:26.928560 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:26.928516    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:27.735851 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:27.735816    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:30.552699 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:30.552667    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:30.653603 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:30.653532    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:31.563255 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:31.563220    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:31.704167 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:31.704134    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:32.165600 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:32.165560    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:32.265474 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:32.265439    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:34.553461 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:34.553428    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:34.765955 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:34.765917    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:37.344310 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:37.344266    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:37.405175 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:37.405130    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:42.162136 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:42.162102    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:42.352559 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:42.352519    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:42.442508 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:42.442422    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:44.954454 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:44.954419    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz"]
Aug 04 03:17:44.957218 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:44.957197    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz"
Aug 04 03:17:44.959349 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:44.959327    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Aug 04 03:17:44.959694 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:44.959677    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2zp7p\""
Aug 04 03:17:44.959755 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:44.959694    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Aug 04 03:17:44.964932 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:44.964903    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz"]
Aug 04 03:17:44.984577 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:44.984529    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4537643e-dee5-49fa-a3b8-56157c00e58e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sjqdz\" (UID: \"4537643e-dee5-49fa-a3b8-56157c00e58e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz"
Aug 04 03:17:44.984734 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:44.984585    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfms7\" (UniqueName: \"kubernetes.io/projected/4537643e-dee5-49fa-a3b8-56157c00e58e-kube-api-access-zfms7\") pod \"openshift-lws-operator-fd8ccff4c-sjqdz\" (UID: \"4537643e-dee5-49fa-a3b8-56157c00e58e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz"
Aug 04 03:17:45.085005 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:45.084973    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4537643e-dee5-49fa-a3b8-56157c00e58e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sjqdz\" (UID: \"4537643e-dee5-49fa-a3b8-56157c00e58e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz"
Aug 04 03:17:45.085154 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:45.085013    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfms7\" (UniqueName: \"kubernetes.io/projected/4537643e-dee5-49fa-a3b8-56157c00e58e-kube-api-access-zfms7\") pod \"openshift-lws-operator-fd8ccff4c-sjqdz\" (UID: \"4537643e-dee5-49fa-a3b8-56157c00e58e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz"
Aug 04 03:17:45.085399 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:45.085374    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4537643e-dee5-49fa-a3b8-56157c00e58e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sjqdz\" (UID: \"4537643e-dee5-49fa-a3b8-56157c00e58e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz"
Aug 04 03:17:45.092733 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:45.092711    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfms7\" (UniqueName: \"kubernetes.io/projected/4537643e-dee5-49fa-a3b8-56157c00e58e-kube-api-access-zfms7\") pod \"openshift-lws-operator-fd8ccff4c-sjqdz\" (UID: \"4537643e-dee5-49fa-a3b8-56157c00e58e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz"
Aug 04 03:17:45.266337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:45.266307    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz"
Aug 04 03:17:45.392068 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:45.392037    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz"]
Aug 04 03:17:45.395290 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:17:45.395260    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4537643e_dee5_49fa_a3b8_56157c00e58e.slice/crio-35c872fb96fbba35b7b110cfaeebe3bf5d88de7063f6c0d14075abad8d40af54 WatchSource:0}: Error finding container 35c872fb96fbba35b7b110cfaeebe3bf5d88de7063f6c0d14075abad8d40af54: Status 404 returned error can't find the container with id 35c872fb96fbba35b7b110cfaeebe3bf5d88de7063f6c0d14075abad8d40af54
Aug 04 03:17:45.796426 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:45.796385    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz" event={"ID":"4537643e-dee5-49fa-a3b8-56157c00e58e","Type":"ContainerStarted","Data":"35c872fb96fbba35b7b110cfaeebe3bf5d88de7063f6c0d14075abad8d40af54"}
Aug 04 03:17:48.812149 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:48.812112    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz" event={"ID":"4537643e-dee5-49fa-a3b8-56157c00e58e","Type":"ContainerStarted","Data":"e22e0fef08d3dcd63ffda0fc3ae0784b88d401f64f9e9550cb8e19c166cec41c"}
Aug 04 03:17:48.827913 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:48.827861    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjqdz" podStartSLOduration=2.349407111 podStartE2EDuration="4.827842941s" totalImagesPullingTime="2.47843583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:17:48.826639588 +0000 UTC m=+486.124780080" watchObservedRunningTime="2026-08-04 03:17:48.827842941 +0000 UTC m=+486.125983428"
Aug 04 03:17:49.448175 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:49.448131    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:49.550840 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:49.550809    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:51.218581 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:51.218530    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:56.172476 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:56.172440    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:17:56.225202 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:17:56.225170    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:03.164949 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:03.164910    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:03.264608 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:03.264562    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:03.784849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:03.784815    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:03.967681 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:03.967641    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:04.077134 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.077049    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:04.126166 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.126135    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"]
Aug 04 03:18:04.129131 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.129113    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.131788 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.131763    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 04 03:18:04.131901 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.131889    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cbjbp\""
Aug 04 03:18:04.132076 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.132064    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 04 03:18:04.132517 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.132504    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 04 03:18:04.132725 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.132712    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 04 03:18:04.144927 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.144897    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"]
Aug 04 03:18:04.230243 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.230197    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d0d4c33-da26-4789-94bc-bbf9cd430b17-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.230243 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.230246    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6d0d4c33-da26-4789-94bc-bbf9cd430b17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.230672 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.230270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6d0d4c33-da26-4789-94bc-bbf9cd430b17-manifests\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.230672 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.230299    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9khm\" (UniqueName: \"kubernetes.io/projected/6d0d4c33-da26-4789-94bc-bbf9cd430b17-kube-api-access-x9khm\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.230672 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.230339    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d0d4c33-da26-4789-94bc-bbf9cd430b17-tmp\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.275991 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.275960    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:04.331228 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.331159    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d0d4c33-da26-4789-94bc-bbf9cd430b17-tmp\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.331228 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.331195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d0d4c33-da26-4789-94bc-bbf9cd430b17-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.331431 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.331363    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6d0d4c33-da26-4789-94bc-bbf9cd430b17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.331431 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.331420    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6d0d4c33-da26-4789-94bc-bbf9cd430b17-manifests\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.331534 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.331468    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9khm\" (UniqueName: \"kubernetes.io/projected/6d0d4c33-da26-4789-94bc-bbf9cd430b17-kube-api-access-x9khm\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.331779 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.331747    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6d0d4c33-da26-4789-94bc-bbf9cd430b17-manifests\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.333651 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.333620    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d0d4c33-da26-4789-94bc-bbf9cd430b17-tmp\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.333758 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.333716    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6d0d4c33-da26-4789-94bc-bbf9cd430b17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.333758 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.333743    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d0d4c33-da26-4789-94bc-bbf9cd430b17-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.340443 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.340412    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9khm\" (UniqueName: \"kubernetes.io/projected/6d0d4c33-da26-4789-94bc-bbf9cd430b17-kube-api-access-x9khm\") pod \"opendatahub-operator-controller-manager-6fb59c9fb6-hjw69\" (UID: \"6d0d4c33-da26-4789-94bc-bbf9cd430b17\") " pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.439018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.438976    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:04.575405 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.575369    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"]
Aug 04 03:18:04.579681 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:18:04.579654    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d0d4c33_da26_4789_94bc_bbf9cd430b17.slice/crio-ce1fefec67970586b0fd6f8a8ff19b3b1a8664623cfa60604c2b94a1a19c44c6 WatchSource:0}: Error finding container ce1fefec67970586b0fd6f8a8ff19b3b1a8664623cfa60604c2b94a1a19c44c6: Status 404 returned error can't find the container with id ce1fefec67970586b0fd6f8a8ff19b3b1a8664623cfa60604c2b94a1a19c44c6
Aug 04 03:18:04.635122 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.635077    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:04.731285 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.731254    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:04.868453 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:04.868367    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69" event={"ID":"6d0d4c33-da26-4789-94bc-bbf9cd430b17","Type":"ContainerStarted","Data":"ce1fefec67970586b0fd6f8a8ff19b3b1a8664623cfa60604c2b94a1a19c44c6"}
Aug 04 03:18:06.688733 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:06.688693    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:06.784337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:06.784299    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:07.881260 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:07.881227    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69" event={"ID":"6d0d4c33-da26-4789-94bc-bbf9cd430b17","Type":"ContainerStarted","Data":"2e0a38cd43383c65223d888edd074195adaf5c8a5985675c465502ee37f05ac1"}
Aug 04 03:18:08.886042 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:08.885954    2559 generic.go:356] "Generic (PLEG): container finished" podID="6d0d4c33-da26-4789-94bc-bbf9cd430b17" containerID="2e0a38cd43383c65223d888edd074195adaf5c8a5985675c465502ee37f05ac1" exitCode=0
Aug 04 03:18:08.886042 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:08.886002    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69" event={"ID":"6d0d4c33-da26-4789-94bc-bbf9cd430b17","Type":"ContainerDied","Data":"2e0a38cd43383c65223d888edd074195adaf5c8a5985675c465502ee37f05ac1"}
Aug 04 03:18:09.417447 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:09.417416    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:09.519900 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:09.519864    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:09.891908 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:09.891871    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69" event={"ID":"6d0d4c33-da26-4789-94bc-bbf9cd430b17","Type":"ContainerStarted","Data":"adce851bb769eaeda434213cea3b911392d014e65d015091af47b50d374a662f"}
Aug 04 03:18:09.892282 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:09.892017    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:09.914781 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:09.914727    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69" podStartSLOduration=1.6316864679999998 podStartE2EDuration="5.914707897s" totalImagesPullingTime="3.278154011s" totalInitContainerRuntime="1.004867418s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:18:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:18:09.912806653 +0000 UTC m=+507.210947140" watchObservedRunningTime="2026-08-04 03:18:09.914707897 +0000 UTC m=+507.212848384"
Aug 04 03:18:19.478919 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:19.478868    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:19.579647 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:19.579607    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:19.905815 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:19.905777    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:19.960246 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:19.960183    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:20.898452 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:20.898420    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6fb59c9fb6-hjw69"
Aug 04 03:18:23.239212 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:23.239156    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:23.348788 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:23.348753    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:23.926909 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:23.926877    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:24.014824 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.014787    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:24.246656 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.246606    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:24.311016 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.310987    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6r9tf"]
Aug 04 03:18:24.322920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.322898    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf"
Aug 04 03:18:24.325991 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.325963    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Aug 04 03:18:24.326293 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.326278    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-jzrn2\""
Aug 04 03:18:24.332620 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.332591    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6r9tf"]
Aug 04 03:18:24.352686 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.352642    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:24.388135 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.388095    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5260a6f3-0749-4ac6-8007-2ad027c6ee97-tls-certs\") pod \"model-serving-api-54d89bd5fc-6r9tf\" (UID: \"5260a6f3-0749-4ac6-8007-2ad027c6ee97\") " pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf"
Aug 04 03:18:24.388318 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.388189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwsgz\" (UniqueName: \"kubernetes.io/projected/5260a6f3-0749-4ac6-8007-2ad027c6ee97-kube-api-access-kwsgz\") pod \"model-serving-api-54d89bd5fc-6r9tf\" (UID: \"5260a6f3-0749-4ac6-8007-2ad027c6ee97\") " pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf"
Aug 04 03:18:24.452960 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.452924    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:24.488830 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.488791    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwsgz\" (UniqueName: \"kubernetes.io/projected/5260a6f3-0749-4ac6-8007-2ad027c6ee97-kube-api-access-kwsgz\") pod \"model-serving-api-54d89bd5fc-6r9tf\" (UID: \"5260a6f3-0749-4ac6-8007-2ad027c6ee97\") " pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf"
Aug 04 03:18:24.488999 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.488846    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5260a6f3-0749-4ac6-8007-2ad027c6ee97-tls-certs\") pod \"model-serving-api-54d89bd5fc-6r9tf\" (UID: \"5260a6f3-0749-4ac6-8007-2ad027c6ee97\") " pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf"
Aug 04 03:18:24.491328 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.491296    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5260a6f3-0749-4ac6-8007-2ad027c6ee97-tls-certs\") pod \"model-serving-api-54d89bd5fc-6r9tf\" (UID: \"5260a6f3-0749-4ac6-8007-2ad027c6ee97\") " pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf"
Aug 04 03:18:24.496732 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.496669    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwsgz\" (UniqueName: \"kubernetes.io/projected/5260a6f3-0749-4ac6-8007-2ad027c6ee97-kube-api-access-kwsgz\") pod \"model-serving-api-54d89bd5fc-6r9tf\" (UID: \"5260a6f3-0749-4ac6-8007-2ad027c6ee97\") " pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf"
Aug 04 03:18:24.634862 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.634817    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf"
Aug 04 03:18:24.767674 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.767479    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6r9tf"]
Aug 04 03:18:24.770946 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:18:24.770908    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5260a6f3_0749_4ac6_8007_2ad027c6ee97.slice/crio-2abe864a600cfb5fd8201bc836860ef625efb47de0626305f7bd7c2624497100 WatchSource:0}: Error finding container 2abe864a600cfb5fd8201bc836860ef625efb47de0626305f7bd7c2624497100: Status 404 returned error can't find the container with id 2abe864a600cfb5fd8201bc836860ef625efb47de0626305f7bd7c2624497100
Aug 04 03:18:24.949298 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:24.949261    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf" event={"ID":"5260a6f3-0749-4ac6-8007-2ad027c6ee97","Type":"ContainerStarted","Data":"2abe864a600cfb5fd8201bc836860ef625efb47de0626305f7bd7c2624497100"}
Aug 04 03:18:26.773303 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:26.773263    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:26.870238 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:26.870203    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:26.870868 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:26.870834    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"]
Aug 04 03:18:26.879390 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:26.879364    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"
Aug 04 03:18:26.882844 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:26.882813    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 04 03:18:26.884949 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:26.883496    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 04 03:18:26.884949 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:26.884384    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 04 03:18:26.884949 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:26.884384    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 04 03:18:26.886678 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:26.886656    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"]
Aug 04 03:18:27.011675 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.011641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab824707-d2bc-4302-af48-16e896a924f3-tls-certs\") pod \"kube-auth-proxy-69dbcc4dfb-8cjct\" (UID: \"ab824707-d2bc-4302-af48-16e896a924f3\") " pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"
Aug 04 03:18:27.011864 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.011700    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab824707-d2bc-4302-af48-16e896a924f3-tmp\") pod \"kube-auth-proxy-69dbcc4dfb-8cjct\" (UID: \"ab824707-d2bc-4302-af48-16e896a924f3\") " pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"
Aug 04 03:18:27.011864 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.011789    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znrvw\" (UniqueName: \"kubernetes.io/projected/ab824707-d2bc-4302-af48-16e896a924f3-kube-api-access-znrvw\") pod \"kube-auth-proxy-69dbcc4dfb-8cjct\" (UID: \"ab824707-d2bc-4302-af48-16e896a924f3\") " pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"
Aug 04 03:18:27.113199 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.113112    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab824707-d2bc-4302-af48-16e896a924f3-tls-certs\") pod \"kube-auth-proxy-69dbcc4dfb-8cjct\" (UID: \"ab824707-d2bc-4302-af48-16e896a924f3\") " pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"
Aug 04 03:18:27.113199 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.113187    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab824707-d2bc-4302-af48-16e896a924f3-tmp\") pod \"kube-auth-proxy-69dbcc4dfb-8cjct\" (UID: \"ab824707-d2bc-4302-af48-16e896a924f3\") " pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"
Aug 04 03:18:27.113478 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.113225    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znrvw\" (UniqueName: \"kubernetes.io/projected/ab824707-d2bc-4302-af48-16e896a924f3-kube-api-access-znrvw\") pod \"kube-auth-proxy-69dbcc4dfb-8cjct\" (UID: \"ab824707-d2bc-4302-af48-16e896a924f3\") " pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"
Aug 04 03:18:27.115720 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.115695    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab824707-d2bc-4302-af48-16e896a924f3-tmp\") pod \"kube-auth-proxy-69dbcc4dfb-8cjct\" (UID: \"ab824707-d2bc-4302-af48-16e896a924f3\") " pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"
Aug 04 03:18:27.115872 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.115853    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab824707-d2bc-4302-af48-16e896a924f3-tls-certs\") pod \"kube-auth-proxy-69dbcc4dfb-8cjct\" (UID: \"ab824707-d2bc-4302-af48-16e896a924f3\") " pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"
Aug 04 03:18:27.120368 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.120345    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znrvw\" (UniqueName: \"kubernetes.io/projected/ab824707-d2bc-4302-af48-16e896a924f3-kube-api-access-znrvw\") pod \"kube-auth-proxy-69dbcc4dfb-8cjct\" (UID: \"ab824707-d2bc-4302-af48-16e896a924f3\") " pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"
Aug 04 03:18:27.198115 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.198037    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"
Aug 04 03:18:27.454927 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.454893    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct"]
Aug 04 03:18:27.457705 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:18:27.457676    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab824707_d2bc_4302_af48_16e896a924f3.slice/crio-145d2a283a01972f7eb28bfd635aa13b9cde934e80c0cce2b92f7e2ae9c06bdc WatchSource:0}: Error finding container 145d2a283a01972f7eb28bfd635aa13b9cde934e80c0cce2b92f7e2ae9c06bdc: Status 404 returned error can't find the container with id 145d2a283a01972f7eb28bfd635aa13b9cde934e80c0cce2b92f7e2ae9c06bdc
Aug 04 03:18:27.910640 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.910607    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"]
Aug 04 03:18:27.913100 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.913082    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:27.915500 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.915474    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Aug 04 03:18:27.915624 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.915500    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Aug 04 03:18:27.915735 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.915719    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Aug 04 03:18:27.915735 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.915726    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-9prf4\""
Aug 04 03:18:27.915939 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.915927    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Aug 04 03:18:27.923144 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.923113    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:27.923845 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.923824    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"]
Aug 04 03:18:27.962959 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.962925    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct" event={"ID":"ab824707-d2bc-4302-af48-16e896a924f3","Type":"ContainerStarted","Data":"145d2a283a01972f7eb28bfd635aa13b9cde934e80c0cce2b92f7e2ae9c06bdc"}
Aug 04 03:18:27.964559 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.964525    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf" event={"ID":"5260a6f3-0749-4ac6-8007-2ad027c6ee97","Type":"ContainerStarted","Data":"b2afefb199f543c0540e940a49b1b7298ec5a68b1920171890583c75e6d79905"}
Aug 04 03:18:27.964723 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.964689    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf"
Aug 04 03:18:27.981008 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.980959    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf" podStartSLOduration=1.3828595940000001 podStartE2EDuration="3.980941714s" totalImagesPullingTime="2.59808212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:18:27.979758377 +0000 UTC m=+525.277898892" watchObservedRunningTime="2026-08-04 03:18:27.980941714 +0000 UTC m=+525.279082202"
Aug 04 03:18:27.993081 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:27.993048    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:28.020110 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.020076    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9184b34d-da03-4eb8-9fae-5bacf8160e0f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.020275 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.020147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.020275 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.020177    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.020275 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.020200    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq8sv\" (UniqueName: \"kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-kube-api-access-xq8sv\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.020376 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.020273    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.020376 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.020314    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.020376 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.020340    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.121668 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.121633    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.121668 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.121677    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9184b34d-da03-4eb8-9fae-5bacf8160e0f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.121913 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.121744    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.122117 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.121769    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.122217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.122157    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq8sv\" (UniqueName: \"kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-kube-api-access-xq8sv\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.122273 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.122217    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.122273 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.122256    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.122935 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.122910    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.124660 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.124633    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.124819 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.124788    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9184b34d-da03-4eb8-9fae-5bacf8160e0f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.124951 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.124930    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.125253 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.125233    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.132830 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.132798    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq8sv\" (UniqueName: \"kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-kube-api-access-xq8sv\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.132979 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.132810    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-t6tld\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.222446 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.222360    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:28.386552 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.386521    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"]
Aug 04 03:18:28.388771 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:18:28.388744    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9184b34d_da03_4eb8_9fae_5bacf8160e0f.slice/crio-0ade10a01f661262ecd008e3fe98b990fc3952d41526f4c685bf464134decafc WatchSource:0}: Error finding container 0ade10a01f661262ecd008e3fe98b990fc3952d41526f4c685bf464134decafc: Status 404 returned error can't find the container with id 0ade10a01f661262ecd008e3fe98b990fc3952d41526f4c685bf464134decafc
Aug 04 03:18:28.808212 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.808180    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:28.970143 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:28.970100    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld" event={"ID":"9184b34d-da03-4eb8-9fae-5bacf8160e0f","Type":"ContainerStarted","Data":"0ade10a01f661262ecd008e3fe98b990fc3952d41526f4c685bf464134decafc"}
Aug 04 03:18:31.098821 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:31.098780    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:18:31.099117 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:31.098847    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:18:31.986381 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:31.986332    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld" event={"ID":"9184b34d-da03-4eb8-9fae-5bacf8160e0f","Type":"ContainerStarted","Data":"03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d"}
Aug 04 03:18:31.986556 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:31.986484    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:31.987933 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:31.987911    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct" event={"ID":"ab824707-d2bc-4302-af48-16e896a924f3","Type":"ContainerStarted","Data":"bcabe5e685a382095f8cc2d722b760df12dc513223a0e4edd9a685547a87c8ad"}
Aug 04 03:18:32.019784 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.019727    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld" podStartSLOduration=2.311842581 podStartE2EDuration="5.019707931s" totalImagesPullingTime="2.70786535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:18:32.018690082 +0000 UTC m=+529.316830609" watchObservedRunningTime="2026-08-04 03:18:32.019707931 +0000 UTC m=+529.317848420"
Aug 04 03:18:32.041465 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.041432    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:32.045156 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.045098    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-69dbcc4dfb-8cjct" podStartSLOduration=2.407077359 podStartE2EDuration="6.045075848s" totalImagesPullingTime="3.637998489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:18:32.043816631 +0000 UTC m=+529.341957140" watchObservedRunningTime="2026-08-04 03:18:32.045075848 +0000 UTC m=+529.343216333"
Aug 04 03:18:32.136409 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.136364    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:32.829789 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.829750    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"]
Aug 04 03:18:32.832236 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.832214    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:32.834813 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.834779    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-849ms\""
Aug 04 03:18:32.848712 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.848682    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"]
Aug 04 03:18:32.864804 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.864764    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:32.965735 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.965693    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:32.965735 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.965734    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:32.965983 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.965753    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:32.965983 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.965820    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tgl9\" (UniqueName: \"kubernetes.io/projected/86bee278-2fe4-430d-a7d9-017944b9590d-kube-api-access-5tgl9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:32.965983 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.965889    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86bee278-2fe4-430d-a7d9-017944b9590d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:32.965983 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.965917    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:32.965983 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.965958    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:32.966195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.966034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86bee278-2fe4-430d-a7d9-017944b9590d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:32.966195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.966059    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86bee278-2fe4-430d-a7d9-017944b9590d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:32.993440 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:32.993417    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:33.067244 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.067210    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86bee278-2fe4-430d-a7d9-017944b9590d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.067424 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.067253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.067424 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.067293    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.067424 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.067349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86bee278-2fe4-430d-a7d9-017944b9590d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.067424 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.067369    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86bee278-2fe4-430d-a7d9-017944b9590d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.067772 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.067748    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.067877 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.067749    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.067877 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.067829    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.067877 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.067858    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.068006 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.067888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tgl9\" (UniqueName: \"kubernetes.io/projected/86bee278-2fe4-430d-a7d9-017944b9590d-kube-api-access-5tgl9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.068390 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.068363    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.068477 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.068389    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.068704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.068679    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.068845 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.068822    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86bee278-2fe4-430d-a7d9-017944b9590d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.070238 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.070215    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86bee278-2fe4-430d-a7d9-017944b9590d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.070428 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.070407    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86bee278-2fe4-430d-a7d9-017944b9590d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.075327 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.075290    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86bee278-2fe4-430d-a7d9-017944b9590d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.075948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.075925    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tgl9\" (UniqueName: \"kubernetes.io/projected/86bee278-2fe4-430d-a7d9-017944b9590d-kube-api-access-5tgl9\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr\" (UID: \"86bee278-2fe4-430d-a7d9-017944b9590d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.142770 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.142682    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:33.201983 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.201940    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:33.280230 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.280207    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"]
Aug 04 03:18:33.282045 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:18:33.282018    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86bee278_2fe4_430d_a7d9_017944b9590d.slice/crio-f0388f2086ca346ccb8e822499fd49a0f896e316b64772ffdedc28c7125d402a WatchSource:0}: Error finding container f0388f2086ca346ccb8e822499fd49a0f896e316b64772ffdedc28c7125d402a: Status 404 returned error can't find the container with id f0388f2086ca346ccb8e822499fd49a0f896e316b64772ffdedc28c7125d402a
Aug 04 03:18:33.996473 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:33.996436    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr" event={"ID":"86bee278-2fe4-430d-a7d9-017944b9590d","Type":"ContainerStarted","Data":"f0388f2086ca346ccb8e822499fd49a0f896e316b64772ffdedc28c7125d402a"}
Aug 04 03:18:34.118061 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:34.118023    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:34.970913 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:34.970840    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"]
Aug 04 03:18:34.973091 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:34.973072    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"
Aug 04 03:18:34.975737 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:34.975712    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4j4tk\""
Aug 04 03:18:34.975879 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:34.975712    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Aug 04 03:18:34.980736 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:34.980707    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"]
Aug 04 03:18:34.985730 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:34.985670    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"
Aug 04 03:18:34.987336 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:34.987309    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"]
Aug 04 03:18:34.989629 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:34.989586    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-lsjjd\""
Aug 04 03:18:34.989755 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:34.989706    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Aug 04 03:18:34.997192 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:34.996935    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"]
Aug 04 03:18:35.064709 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.064665    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:35.086643 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.086605    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/023e8e85-42fc-4303-9018-b02effcbfd18-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6z7g6\" (UID: \"023e8e85-42fc-4303-9018-b02effcbfd18\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"
Aug 04 03:18:35.086830 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.086671    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/327e7889-7bb5-4bac-8c54-929b09255d7f-cert\") pod \"kserve-controller-manager-d6f4b65b8-xhgpl\" (UID: \"327e7889-7bb5-4bac-8c54-929b09255d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"
Aug 04 03:18:35.086830 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.086695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brc6n\" (UniqueName: \"kubernetes.io/projected/327e7889-7bb5-4bac-8c54-929b09255d7f-kube-api-access-brc6n\") pod \"kserve-controller-manager-d6f4b65b8-xhgpl\" (UID: \"327e7889-7bb5-4bac-8c54-929b09255d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"
Aug 04 03:18:35.086830 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.086767    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz95v\" (UniqueName: \"kubernetes.io/projected/023e8e85-42fc-4303-9018-b02effcbfd18-kube-api-access-gz95v\") pod \"llmisvc-controller-manager-777c6bd7d5-6z7g6\" (UID: \"023e8e85-42fc-4303-9018-b02effcbfd18\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"
Aug 04 03:18:35.166378 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.166336    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:35.187771 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.187728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz95v\" (UniqueName: \"kubernetes.io/projected/023e8e85-42fc-4303-9018-b02effcbfd18-kube-api-access-gz95v\") pod \"llmisvc-controller-manager-777c6bd7d5-6z7g6\" (UID: \"023e8e85-42fc-4303-9018-b02effcbfd18\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"
Aug 04 03:18:35.187972 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.187839    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/023e8e85-42fc-4303-9018-b02effcbfd18-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6z7g6\" (UID: \"023e8e85-42fc-4303-9018-b02effcbfd18\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"
Aug 04 03:18:35.187972 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.187888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/327e7889-7bb5-4bac-8c54-929b09255d7f-cert\") pod \"kserve-controller-manager-d6f4b65b8-xhgpl\" (UID: \"327e7889-7bb5-4bac-8c54-929b09255d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"
Aug 04 03:18:35.187972 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.187918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brc6n\" (UniqueName: \"kubernetes.io/projected/327e7889-7bb5-4bac-8c54-929b09255d7f-kube-api-access-brc6n\") pod \"kserve-controller-manager-d6f4b65b8-xhgpl\" (UID: \"327e7889-7bb5-4bac-8c54-929b09255d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"
Aug 04 03:18:35.188144 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:18:35.188009    2559 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Aug 04 03:18:35.188144 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:18:35.188079    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/327e7889-7bb5-4bac-8c54-929b09255d7f-cert podName:327e7889-7bb5-4bac-8c54-929b09255d7f nodeName:}" failed. No retries permitted until 2026-08-04 03:18:35.688057015 +0000 UTC m=+532.986197484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/327e7889-7bb5-4bac-8c54-929b09255d7f-cert") pod "kserve-controller-manager-d6f4b65b8-xhgpl" (UID: "327e7889-7bb5-4bac-8c54-929b09255d7f") : secret "kserve-webhook-server-cert" not found
Aug 04 03:18:35.190590 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.190563    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/023e8e85-42fc-4303-9018-b02effcbfd18-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6z7g6\" (UID: \"023e8e85-42fc-4303-9018-b02effcbfd18\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"
Aug 04 03:18:35.202363 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.202328    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brc6n\" (UniqueName: \"kubernetes.io/projected/327e7889-7bb5-4bac-8c54-929b09255d7f-kube-api-access-brc6n\") pod \"kserve-controller-manager-d6f4b65b8-xhgpl\" (UID: \"327e7889-7bb5-4bac-8c54-929b09255d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"
Aug 04 03:18:35.202515 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.202363    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz95v\" (UniqueName: \"kubernetes.io/projected/023e8e85-42fc-4303-9018-b02effcbfd18-kube-api-access-gz95v\") pod \"llmisvc-controller-manager-777c6bd7d5-6z7g6\" (UID: \"023e8e85-42fc-4303-9018-b02effcbfd18\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"
Aug 04 03:18:35.262128 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.262071    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:35.299824 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.299785    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"
Aug 04 03:18:35.360130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.360095    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:35.693013 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.692930    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/327e7889-7bb5-4bac-8c54-929b09255d7f-cert\") pod \"kserve-controller-manager-d6f4b65b8-xhgpl\" (UID: \"327e7889-7bb5-4bac-8c54-929b09255d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"
Aug 04 03:18:35.695646 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.695616    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/327e7889-7bb5-4bac-8c54-929b09255d7f-cert\") pod \"kserve-controller-manager-d6f4b65b8-xhgpl\" (UID: \"327e7889-7bb5-4bac-8c54-929b09255d7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"
Aug 04 03:18:35.886632 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:35.886595    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"
Aug 04 03:18:36.150899 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.150856    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:18:36.151316 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.150945    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:18:36.151316 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.150996    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:18:36.297775 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.297741    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"]
Aug 04 03:18:36.300281 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:18:36.300238    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod327e7889_7bb5_4bac_8c54_929b09255d7f.slice/crio-789770f188af650b9d09d722470063e0cf612f9a80c01a43b74a7d65fff0deeb WatchSource:0}: Error finding container 789770f188af650b9d09d722470063e0cf612f9a80c01a43b74a7d65fff0deeb: Status 404 returned error can't find the container with id 789770f188af650b9d09d722470063e0cf612f9a80c01a43b74a7d65fff0deeb
Aug 04 03:18:36.321633 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.321522    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"]
Aug 04 03:18:36.324308 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:18:36.324278    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod023e8e85_42fc_4303_9018_b02effcbfd18.slice/crio-1eb1cc8671eaffbac084aab45b39e102b8b07055b39e14b0134c70708665bcf6 WatchSource:0}: Error finding container 1eb1cc8671eaffbac084aab45b39e102b8b07055b39e14b0134c70708665bcf6: Status 404 returned error can't find the container with id 1eb1cc8671eaffbac084aab45b39e102b8b07055b39e14b0134c70708665bcf6
Aug 04 03:18:36.550966 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.550930    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"]
Aug 04 03:18:36.551180 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.551154    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld" podUID="9184b34d-da03-4eb8-9fae-5bacf8160e0f" containerName="discovery" containerID="cri-o://03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d" gracePeriod=30
Aug 04 03:18:36.782112 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.782083    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:36.905786 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.905702    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-cacerts\") pod \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") "
Aug 04 03:18:36.905948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.905755    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq8sv\" (UniqueName: \"kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-kube-api-access-xq8sv\") pod \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") "
Aug 04 03:18:36.905948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.905856    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-ca-configmap\") pod \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") "
Aug 04 03:18:36.905948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.905920    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9184b34d-da03-4eb8-9fae-5bacf8160e0f-local-certs\") pod \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") "
Aug 04 03:18:36.906118 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.905970    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-kubeconfig\") pod \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") "
Aug 04 03:18:36.906118 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.905998    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-dns-cert\") pod \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") "
Aug 04 03:18:36.906118 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.906028    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-token\") pod \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\" (UID: \"9184b34d-da03-4eb8-9fae-5bacf8160e0f\") "
Aug 04 03:18:36.906278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.906219    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "9184b34d-da03-4eb8-9fae-5bacf8160e0f" (UID: "9184b34d-da03-4eb8-9fae-5bacf8160e0f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:18:36.908589 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.908554    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "9184b34d-da03-4eb8-9fae-5bacf8160e0f" (UID: "9184b34d-da03-4eb8-9fae-5bacf8160e0f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 03:18:36.908701 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.908619    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-cacerts" (OuterVolumeSpecName: "cacerts") pod "9184b34d-da03-4eb8-9fae-5bacf8160e0f" (UID: "9184b34d-da03-4eb8-9fae-5bacf8160e0f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 03:18:36.908757 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.908711    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "9184b34d-da03-4eb8-9fae-5bacf8160e0f" (UID: "9184b34d-da03-4eb8-9fae-5bacf8160e0f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 03:18:36.908757 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.908718    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9184b34d-da03-4eb8-9fae-5bacf8160e0f-local-certs" (OuterVolumeSpecName: "local-certs") pod "9184b34d-da03-4eb8-9fae-5bacf8160e0f" (UID: "9184b34d-da03-4eb8-9fae-5bacf8160e0f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:18:36.908757 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.908728    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-kube-api-access-xq8sv" (OuterVolumeSpecName: "kube-api-access-xq8sv") pod "9184b34d-da03-4eb8-9fae-5bacf8160e0f" (UID: "9184b34d-da03-4eb8-9fae-5bacf8160e0f"). InnerVolumeSpecName "kube-api-access-xq8sv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:18:36.908862 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:36.908750    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-token" (OuterVolumeSpecName: "istio-token") pod "9184b34d-da03-4eb8-9fae-5bacf8160e0f" (UID: "9184b34d-da03-4eb8-9fae-5bacf8160e0f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:18:37.006650 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.006597    2559 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-cacerts\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:18:37.006650 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.006648    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq8sv\" (UniqueName: \"kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-kube-api-access-xq8sv\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:18:37.006829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.006661    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-ca-configmap\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:18:37.006829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.006671    2559 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9184b34d-da03-4eb8-9fae-5bacf8160e0f-local-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:18:37.006829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.006680    2559 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-kubeconfig\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:18:37.006829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.006687    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-csr-dns-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:18:37.006829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.006695    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9184b34d-da03-4eb8-9fae-5bacf8160e0f-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:18:37.010888 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.010861    2559 generic.go:356] "Generic (PLEG): container finished" podID="9184b34d-da03-4eb8-9fae-5bacf8160e0f" containerID="03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d" exitCode=0
Aug 04 03:18:37.011020 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.010906    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"
Aug 04 03:18:37.011020 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.010944    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld" event={"ID":"9184b34d-da03-4eb8-9fae-5bacf8160e0f","Type":"ContainerDied","Data":"03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d"}
Aug 04 03:18:37.011020 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.010996    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld" event={"ID":"9184b34d-da03-4eb8-9fae-5bacf8160e0f","Type":"ContainerDied","Data":"0ade10a01f661262ecd008e3fe98b990fc3952d41526f4c685bf464134decafc"}
Aug 04 03:18:37.011148 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.011025    2559 scope.go:117] "RemoveContainer" containerID="03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d"
Aug 04 03:18:37.012403 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.012377    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl" event={"ID":"327e7889-7bb5-4bac-8c54-929b09255d7f","Type":"ContainerStarted","Data":"789770f188af650b9d09d722470063e0cf612f9a80c01a43b74a7d65fff0deeb"}
Aug 04 03:18:37.013613 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.013578    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6" event={"ID":"023e8e85-42fc-4303-9018-b02effcbfd18","Type":"ContainerStarted","Data":"1eb1cc8671eaffbac084aab45b39e102b8b07055b39e14b0134c70708665bcf6"}
Aug 04 03:18:37.014966 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.014943    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr" event={"ID":"86bee278-2fe4-430d-a7d9-017944b9590d","Type":"ContainerStarted","Data":"be4ba2beade84485a2d254926b9d76bb4781f360d3436f28ace91f13fd5fe280"}
Aug 04 03:18:37.019876 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.019838    2559 scope.go:117] "RemoveContainer" containerID="03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d"
Aug 04 03:18:37.020468 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:18:37.020436    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d\": container with ID starting with 03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d not found: ID does not exist" containerID="03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d"
Aug 04 03:18:37.020582 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.020480    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d"} err="failed to get container status \"03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d\": rpc error: code = NotFound desc = could not find container \"03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d\": container with ID starting with 03688d6b7a9c4dc728c265a8acb2c6e43e02cd3a4be5f8ad3dec67832ee6455d not found: ID does not exist"
Aug 04 03:18:37.056385 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.056333    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr" podStartSLOduration=2.189822852 podStartE2EDuration="5.056315552s" totalImagesPullingTime="2.8664927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:18:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:18:37.055304039 +0000 UTC m=+534.353444524" watchObservedRunningTime="2026-08-04 03:18:37.056315552 +0000 UTC m=+534.354456039"
Aug 04 03:18:37.070281 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.070249    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"]
Aug 04 03:18:37.074920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.074895    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-t6tld"]
Aug 04 03:18:37.142994 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.142888    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:37.148774 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.148734    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:37.240850 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:37.240818    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9184b34d-da03-4eb8-9fae-5bacf8160e0f" path="/var/lib/kubelet/pods/9184b34d-da03-4eb8-9fae-5bacf8160e0f/volumes"
Aug 04 03:18:38.021144 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:38.021110    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:38.022043 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:38.022023    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr"
Aug 04 03:18:38.360022 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:38.359936    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:38.469433 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:38.469391    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:18:38.975739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:38.975698    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6r9tf"
Aug 04 03:18:40.032098 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:40.032058    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl" event={"ID":"327e7889-7bb5-4bac-8c54-929b09255d7f","Type":"ContainerStarted","Data":"c6b51b556e43f0039ed8b237930c0b0430dd04e5a300fd51fcc0bffa1149ad95"}
Aug 04 03:18:40.032613 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:40.032162    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"
Aug 04 03:18:40.034086 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:40.034053    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6" event={"ID":"023e8e85-42fc-4303-9018-b02effcbfd18","Type":"ContainerStarted","Data":"bb8306a41b5a1d1f8a64f29e32989a0a9285503c2c38dfdea69a57a6f9a3c539"}
Aug 04 03:18:40.034302 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:40.034284    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"
Aug 04 03:18:40.048565 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:40.048401    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl" podStartSLOduration=2.979392022 podStartE2EDuration="6.048381036s" totalImagesPullingTime="3.068989014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:18:40.047719544 +0000 UTC m=+537.345860032" watchObservedRunningTime="2026-08-04 03:18:40.048381036 +0000 UTC m=+537.346521530"
Aug 04 03:18:40.064252 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:18:40.064202    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6" podStartSLOduration=3.016099212 podStartE2EDuration="6.064184669s" totalImagesPullingTime="3.048085457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:18:40.062688974 +0000 UTC m=+537.360829490" watchObservedRunningTime="2026-08-04 03:18:40.064184669 +0000 UTC m=+537.362325156"
Aug 04 03:19:07.282497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:07.282442    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:11.039762 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:11.039722    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6z7g6"
Aug 04 03:19:11.043249 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:11.043226    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xhgpl"
Aug 04 03:19:24.671752 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.671711    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:24.774092 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.774053    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:24.797022 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.796992    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zqrtn"]
Aug 04 03:19:24.797296 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.797285    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9184b34d-da03-4eb8-9fae-5bacf8160e0f" containerName="discovery"
Aug 04 03:19:24.797296 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.797297    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9184b34d-da03-4eb8-9fae-5bacf8160e0f" containerName="discovery"
Aug 04 03:19:24.797370 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.797350    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9184b34d-da03-4eb8-9fae-5bacf8160e0f" containerName="discovery"
Aug 04 03:19:24.804307 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.804278    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zqrtn"
Aug 04 03:19:24.806880 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.806851    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 04 03:19:24.807935 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.807754    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 04 03:19:24.807935 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.807809    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-78vtm\""
Aug 04 03:19:24.809830 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.809802    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zqrtn"]
Aug 04 03:19:24.887476 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.887441    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:24.905838 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:24.905802    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw9bc\" (UniqueName: \"kubernetes.io/projected/52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0-kube-api-access-jw9bc\") pod \"kuadrant-operator-catalog-zqrtn\" (UID: \"52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0\") " pod="kuadrant-system/kuadrant-operator-catalog-zqrtn"
Aug 04 03:19:25.006818 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.006775    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw9bc\" (UniqueName: \"kubernetes.io/projected/52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0-kube-api-access-jw9bc\") pod \"kuadrant-operator-catalog-zqrtn\" (UID: \"52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0\") " pod="kuadrant-system/kuadrant-operator-catalog-zqrtn"
Aug 04 03:19:25.015324 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.015295    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw9bc\" (UniqueName: \"kubernetes.io/projected/52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0-kube-api-access-jw9bc\") pod \"kuadrant-operator-catalog-zqrtn\" (UID: \"52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0\") " pod="kuadrant-system/kuadrant-operator-catalog-zqrtn"
Aug 04 03:19:25.115709 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.115670    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zqrtn"
Aug 04 03:19:25.230623 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.230589    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zqrtn"]
Aug 04 03:19:25.250767 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.250741    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zqrtn"]
Aug 04 03:19:25.253392 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:19:25.253365    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d8e6ca_cbc0_4c98_9f2d_d3784eef64d0.slice/crio-9bd29d92d7da7a513ec9d170985b279eeed0435adfac8c5e135a742d6e871529 WatchSource:0}: Error finding container 9bd29d92d7da7a513ec9d170985b279eeed0435adfac8c5e135a742d6e871529: Status 404 returned error can't find the container with id 9bd29d92d7da7a513ec9d170985b279eeed0435adfac8c5e135a742d6e871529
Aug 04 03:19:25.372950 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.372908    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zfsc4"]
Aug 04 03:19:25.375922 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.375897    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zfsc4"
Aug 04 03:19:25.388486 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.388458    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zfsc4"]
Aug 04 03:19:25.511961 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.511873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tvqm\" (UniqueName: \"kubernetes.io/projected/79e97959-2959-4519-a3e3-d019d099070d-kube-api-access-6tvqm\") pod \"kuadrant-operator-catalog-zfsc4\" (UID: \"79e97959-2959-4519-a3e3-d019d099070d\") " pod="kuadrant-system/kuadrant-operator-catalog-zfsc4"
Aug 04 03:19:25.612701 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.612658    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tvqm\" (UniqueName: \"kubernetes.io/projected/79e97959-2959-4519-a3e3-d019d099070d-kube-api-access-6tvqm\") pod \"kuadrant-operator-catalog-zfsc4\" (UID: \"79e97959-2959-4519-a3e3-d019d099070d\") " pod="kuadrant-system/kuadrant-operator-catalog-zfsc4"
Aug 04 03:19:25.620024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.620002    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tvqm\" (UniqueName: \"kubernetes.io/projected/79e97959-2959-4519-a3e3-d019d099070d-kube-api-access-6tvqm\") pod \"kuadrant-operator-catalog-zfsc4\" (UID: \"79e97959-2959-4519-a3e3-d019d099070d\") " pod="kuadrant-system/kuadrant-operator-catalog-zfsc4"
Aug 04 03:19:25.685617 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.685579    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zfsc4"
Aug 04 03:19:25.814221 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:25.814198    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zfsc4"]
Aug 04 03:19:25.816601 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:19:25.816575    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e97959_2959_4519_a3e3_d019d099070d.slice/crio-7867cc8a153f0621022b5b30e0512e605185cb647b5445e9a1e235e44edefa49 WatchSource:0}: Error finding container 7867cc8a153f0621022b5b30e0512e605185cb647b5445e9a1e235e44edefa49: Status 404 returned error can't find the container with id 7867cc8a153f0621022b5b30e0512e605185cb647b5445e9a1e235e44edefa49
Aug 04 03:19:26.222297 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:26.222205    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zfsc4" event={"ID":"79e97959-2959-4519-a3e3-d019d099070d","Type":"ContainerStarted","Data":"7867cc8a153f0621022b5b30e0512e605185cb647b5445e9a1e235e44edefa49"}
Aug 04 03:19:26.223828 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:26.223791    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zqrtn" event={"ID":"52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0","Type":"ContainerStarted","Data":"9bd29d92d7da7a513ec9d170985b279eeed0435adfac8c5e135a742d6e871529"}
Aug 04 03:19:28.233389 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:28.233358    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zfsc4" event={"ID":"79e97959-2959-4519-a3e3-d019d099070d","Type":"ContainerStarted","Data":"2d7a73a5020a13376a7901d37b561739af3773138acfebec8bcd75bcd8027851"}
Aug 04 03:19:28.234833 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:28.234810    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zqrtn" event={"ID":"52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0","Type":"ContainerStarted","Data":"868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396"}
Aug 04 03:19:28.234940 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:28.234875    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-zqrtn" podUID="52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0" containerName="registry-server" containerID="cri-o://868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396" gracePeriod=2
Aug 04 03:19:28.248375 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:28.248324    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-zfsc4" podStartSLOduration=1.745982674 podStartE2EDuration="3.248305362s" totalImagesPullingTime="1.502322688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:19:28.247132573 +0000 UTC m=+585.545273080" watchObservedRunningTime="2026-08-04 03:19:28.248305362 +0000 UTC m=+585.546445850"
Aug 04 03:19:28.261561 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:28.261494    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-zqrtn" podStartSLOduration=2.196414925 podStartE2EDuration="4.261477826s" totalImagesPullingTime="2.065062901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:19:28.260500146 +0000 UTC m=+585.558640633" watchObservedRunningTime="2026-08-04 03:19:28.261477826 +0000 UTC m=+585.559618314"
Aug 04 03:19:28.484265 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:28.484200    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zqrtn"
Aug 04 03:19:28.635401 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:28.635353    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw9bc\" (UniqueName: \"kubernetes.io/projected/52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0-kube-api-access-jw9bc\") pod \"52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0\" (UID: \"52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0\") "
Aug 04 03:19:28.637577 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:28.637532    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0-kube-api-access-jw9bc" (OuterVolumeSpecName: "kube-api-access-jw9bc") pod "52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0" (UID: "52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0"). InnerVolumeSpecName "kube-api-access-jw9bc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:19:28.736459 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:28.736360    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw9bc\" (UniqueName: \"kubernetes.io/projected/52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0-kube-api-access-jw9bc\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:19:29.239322 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:29.239279    2559 generic.go:356] "Generic (PLEG): container finished" podID="52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0" containerID="868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396" exitCode=0
Aug 04 03:19:29.239835 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:29.239337    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zqrtn"
Aug 04 03:19:29.239835 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:29.239365    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zqrtn" event={"ID":"52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0","Type":"ContainerDied","Data":"868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396"}
Aug 04 03:19:29.239835 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:29.239413    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zqrtn" event={"ID":"52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0","Type":"ContainerDied","Data":"9bd29d92d7da7a513ec9d170985b279eeed0435adfac8c5e135a742d6e871529"}
Aug 04 03:19:29.239835 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:29.239432    2559 scope.go:117] "RemoveContainer" containerID="868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396"
Aug 04 03:19:29.248789 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:29.248771    2559 scope.go:117] "RemoveContainer" containerID="868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396"
Aug 04 03:19:29.249060 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:19:29.249039    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396\": container with ID starting with 868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396 not found: ID does not exist" containerID="868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396"
Aug 04 03:19:29.249122 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:29.249072    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396"} err="failed to get container status \"868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396\": rpc error: code = NotFound desc = could not find container \"868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396\": container with ID starting with 868440a936df134f9278cd6906cfa8c90005535c384def4358f9b490adc0e396 not found: ID does not exist"
Aug 04 03:19:29.259318 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:29.259290    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zqrtn"]
Aug 04 03:19:29.262427 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:29.262404    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zqrtn"]
Aug 04 03:19:31.237111 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:31.237073    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0" path="/var/lib/kubelet/pods/52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0/volumes"
Aug 04 03:19:35.686667 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:35.686626    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-zfsc4"
Aug 04 03:19:35.687105 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:35.686692    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-zfsc4"
Aug 04 03:19:35.708632 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:35.708592    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-zfsc4"
Aug 04 03:19:36.289075 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:36.289047    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-zfsc4"
Aug 04 03:19:42.324815 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:42.324775    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:42.391218 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:42.391181    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:46.899298 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:46.899264    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:47.046752 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:47.046720    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:47.146878 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:47.146844    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:47.645151 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:47.645101    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:47.693233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:47.693193    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:52.915935 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:52.915898    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:53.517531 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:53.517491    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:56.971842 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:56.971807    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fg85w"]
Aug 04 03:19:56.972226 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:56.972100    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0" containerName="registry-server"
Aug 04 03:19:56.972226 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:56.972110    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0" containerName="registry-server"
Aug 04 03:19:56.972226 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:56.972170    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="52d8e6ca-cbc0-4c98-9f2d-d3784eef64d0" containerName="registry-server"
Aug 04 03:19:56.975207 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:56.975191    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fg85w"
Aug 04 03:19:56.977289 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:56.977262    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-dqtjb\""
Aug 04 03:19:56.983873 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:56.983845    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fg85w"]
Aug 04 03:19:57.064483 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:57.064445    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5vp8\" (UniqueName: \"kubernetes.io/projected/4bb35e85-467d-448a-a0e2-e4a44cc7383d-kube-api-access-f5vp8\") pod \"authorino-operator-75cc8d5cb7-fg85w\" (UID: \"4bb35e85-467d-448a-a0e2-e4a44cc7383d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fg85w"
Aug 04 03:19:57.164916 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:57.164873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5vp8\" (UniqueName: \"kubernetes.io/projected/4bb35e85-467d-448a-a0e2-e4a44cc7383d-kube-api-access-f5vp8\") pod \"authorino-operator-75cc8d5cb7-fg85w\" (UID: \"4bb35e85-467d-448a-a0e2-e4a44cc7383d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fg85w"
Aug 04 03:19:57.178645 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:57.178615    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5vp8\" (UniqueName: \"kubernetes.io/projected/4bb35e85-467d-448a-a0e2-e4a44cc7383d-kube-api-access-f5vp8\") pod \"authorino-operator-75cc8d5cb7-fg85w\" (UID: \"4bb35e85-467d-448a-a0e2-e4a44cc7383d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fg85w"
Aug 04 03:19:57.285291 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:57.285255    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fg85w"
Aug 04 03:19:57.421495 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:57.421336    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fg85w"]
Aug 04 03:19:57.424345 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:19:57.424317    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb35e85_467d_448a_a0e2_e4a44cc7383d.slice/crio-66d9a51c24a499fa8a2b1dd17fd229a5404503ed49a0705d347437dc02bba2bc WatchSource:0}: Error finding container 66d9a51c24a499fa8a2b1dd17fd229a5404503ed49a0705d347437dc02bba2bc: Status 404 returned error can't find the container with id 66d9a51c24a499fa8a2b1dd17fd229a5404503ed49a0705d347437dc02bba2bc
Aug 04 03:19:57.453369 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:57.453331    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:57.543414 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:57.543306    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:57.930017 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:57.929917    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:19:58.361963 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:19:58.361916    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fg85w" event={"ID":"4bb35e85-467d-448a-a0e2-e4a44cc7383d","Type":"ContainerStarted","Data":"66d9a51c24a499fa8a2b1dd17fd229a5404503ed49a0705d347437dc02bba2bc"}
Aug 04 03:20:00.163257 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:00.163162    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:00.264567 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:00.264507    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:00.372657 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:00.372620    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fg85w" event={"ID":"4bb35e85-467d-448a-a0e2-e4a44cc7383d","Type":"ContainerStarted","Data":"5df3f8cd5b1b91732c90fcdc5fb9c326205d9acf2fc251f9ba6cf30dc513cb5a"}
Aug 04 03:20:00.372843 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:00.372744    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fg85w"
Aug 04 03:20:00.398209 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:00.398159    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fg85w" podStartSLOduration=2.062915898 podStartE2EDuration="4.398142473s" totalImagesPullingTime="2.335226575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:20:00.396670474 +0000 UTC m=+617.694810964" watchObservedRunningTime="2026-08-04 03:20:00.398142473 +0000 UTC m=+617.696283007"
Aug 04 03:20:02.036180 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.036149    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"]
Aug 04 03:20:02.040731 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.040709    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:02.043405 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.043384    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rgf8h\""
Aug 04 03:20:02.052017 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.051993    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"]
Aug 04 03:20:02.110223 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.110184    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82kv4\" (UniqueName: \"kubernetes.io/projected/d101a74d-4c2b-446b-85b4-5c47ddf22d28-kube-api-access-82kv4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" (UID: \"d101a74d-4c2b-446b-85b4-5c47ddf22d28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:02.110385 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.110255    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d101a74d-4c2b-446b-85b4-5c47ddf22d28-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" (UID: \"d101a74d-4c2b-446b-85b4-5c47ddf22d28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:02.210919 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.210878    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82kv4\" (UniqueName: \"kubernetes.io/projected/d101a74d-4c2b-446b-85b4-5c47ddf22d28-kube-api-access-82kv4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" (UID: \"d101a74d-4c2b-446b-85b4-5c47ddf22d28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:02.211087 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.210932    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d101a74d-4c2b-446b-85b4-5c47ddf22d28-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" (UID: \"d101a74d-4c2b-446b-85b4-5c47ddf22d28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:02.211293 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.211276    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d101a74d-4c2b-446b-85b4-5c47ddf22d28-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" (UID: \"d101a74d-4c2b-446b-85b4-5c47ddf22d28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:02.219634 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.219610    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82kv4\" (UniqueName: \"kubernetes.io/projected/d101a74d-4c2b-446b-85b4-5c47ddf22d28-kube-api-access-82kv4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" (UID: \"d101a74d-4c2b-446b-85b4-5c47ddf22d28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:02.350493 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.350401    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:02.482302 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.482264    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"]
Aug 04 03:20:02.485790 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:20:02.485762    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd101a74d_4c2b_446b_85b4_5c47ddf22d28.slice/crio-de92997dc405901a9ffc8de13af341a9ab753b7c962d9c379125ce00e2c364fb WatchSource:0}: Error finding container de92997dc405901a9ffc8de13af341a9ab753b7c962d9c379125ce00e2c364fb: Status 404 returned error can't find the container with id de92997dc405901a9ffc8de13af341a9ab753b7c962d9c379125ce00e2c364fb
Aug 04 03:20:02.806476 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:02.806440    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:03.134330 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:03.134236    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:03.385201 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:03.385129    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7" event={"ID":"d101a74d-4c2b-446b-85b4-5c47ddf22d28","Type":"ContainerStarted","Data":"de92997dc405901a9ffc8de13af341a9ab753b7c962d9c379125ce00e2c364fb"}
Aug 04 03:20:04.924985 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:04.924885    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6"]
Aug 04 03:20:04.927246 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:04.927223    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6"
Aug 04 03:20:04.929604 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:04.929583    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-mhhrl\""
Aug 04 03:20:04.929725 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:04.929626    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Aug 04 03:20:04.939099 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:04.939075    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6"]
Aug 04 03:20:05.034940 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:05.034906    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tz6b\" (UniqueName: \"kubernetes.io/projected/fafddda8-c6d9-4964-b15b-6f61a772fa1d-kube-api-access-5tz6b\") pod \"dns-operator-controller-manager-864dd78c4c-4fmw6\" (UID: \"fafddda8-c6d9-4964-b15b-6f61a772fa1d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6"
Aug 04 03:20:05.135361 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:05.135318    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tz6b\" (UniqueName: \"kubernetes.io/projected/fafddda8-c6d9-4964-b15b-6f61a772fa1d-kube-api-access-5tz6b\") pod \"dns-operator-controller-manager-864dd78c4c-4fmw6\" (UID: \"fafddda8-c6d9-4964-b15b-6f61a772fa1d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6"
Aug 04 03:20:05.147396 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:05.147370    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tz6b\" (UniqueName: \"kubernetes.io/projected/fafddda8-c6d9-4964-b15b-6f61a772fa1d-kube-api-access-5tz6b\") pod \"dns-operator-controller-manager-864dd78c4c-4fmw6\" (UID: \"fafddda8-c6d9-4964-b15b-6f61a772fa1d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6"
Aug 04 03:20:05.238047 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:05.238016    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6"
Aug 04 03:20:05.381288 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:05.381267    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6"]
Aug 04 03:20:05.383356 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:20:05.383323    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfafddda8_c6d9_4964_b15b_6f61a772fa1d.slice/crio-00b6dfc9e57817be56383cef7eaf638eb39b663ee707e985782bab4f97af98da WatchSource:0}: Error finding container 00b6dfc9e57817be56383cef7eaf638eb39b663ee707e985782bab4f97af98da: Status 404 returned error can't find the container with id 00b6dfc9e57817be56383cef7eaf638eb39b663ee707e985782bab4f97af98da
Aug 04 03:20:05.394472 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:05.394447    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6" event={"ID":"fafddda8-c6d9-4964-b15b-6f61a772fa1d","Type":"ContainerStarted","Data":"00b6dfc9e57817be56383cef7eaf638eb39b663ee707e985782bab4f97af98da"}
Aug 04 03:20:05.524021 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:05.523936    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:06.210222 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:06.210167    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:07.328529 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:07.328491    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"]
Aug 04 03:20:07.331294 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:07.331230    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:07.333810 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:07.333715    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-tkt4r\""
Aug 04 03:20:07.344471 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:07.343307    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"]
Aug 04 03:20:07.405715 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:07.405670    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6" event={"ID":"fafddda8-c6d9-4964-b15b-6f61a772fa1d","Type":"ContainerStarted","Data":"68e2a06dccc760c254b8dfd4d9be57729181d5616298d92a33790e1445d305cd"}
Aug 04 03:20:07.405900 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:07.405756    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6"
Aug 04 03:20:07.423008 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:07.422893    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6" podStartSLOduration=1.644977304 podStartE2EDuration="3.422871588s" totalImagesPullingTime="1.777894284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:20:07.421954478 +0000 UTC m=+624.720094988" watchObservedRunningTime="2026-08-04 03:20:07.422871588 +0000 UTC m=+624.721012077"
Aug 04 03:20:07.457100 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:07.457054    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2mbw\" (UniqueName: \"kubernetes.io/projected/673df7a3-2a15-4067-af62-0ca57549b671-kube-api-access-g2mbw\") pod \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" (UID: \"673df7a3-2a15-4067-af62-0ca57549b671\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:07.557579 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:07.557511    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2mbw\" (UniqueName: \"kubernetes.io/projected/673df7a3-2a15-4067-af62-0ca57549b671-kube-api-access-g2mbw\") pod \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" (UID: \"673df7a3-2a15-4067-af62-0ca57549b671\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:07.566258 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:07.566227    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2mbw\" (UniqueName: \"kubernetes.io/projected/673df7a3-2a15-4067-af62-0ca57549b671-kube-api-access-g2mbw\") pod \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" (UID: \"673df7a3-2a15-4067-af62-0ca57549b671\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:07.645253 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:07.645212    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:08.589278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:08.589253    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"]
Aug 04 03:20:08.591338 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:20:08.591312    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673df7a3_2a15_4067_af62_0ca57549b671.slice/crio-c7417724c5f0de16da8eef4a154c6e31b3266b87c2765a007686ad428da4e9b5 WatchSource:0}: Error finding container c7417724c5f0de16da8eef4a154c6e31b3266b87c2765a007686ad428da4e9b5: Status 404 returned error can't find the container with id c7417724c5f0de16da8eef4a154c6e31b3266b87c2765a007686ad428da4e9b5
Aug 04 03:20:08.621288 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:08.621258    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:09.318128 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:09.318093    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:09.378098 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:09.378066    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:09.416407 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:09.416369    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f" event={"ID":"673df7a3-2a15-4067-af62-0ca57549b671","Type":"ContainerStarted","Data":"c7417724c5f0de16da8eef4a154c6e31b3266b87c2765a007686ad428da4e9b5"}
Aug 04 03:20:09.417875 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:09.417851    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7" event={"ID":"d101a74d-4c2b-446b-85b4-5c47ddf22d28","Type":"ContainerStarted","Data":"a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50"}
Aug 04 03:20:09.418014 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:09.417973    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:09.436830 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:09.436781    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7" podStartSLOduration=1.428922589 podStartE2EDuration="7.436764386s" totalImagesPullingTime="6.007841797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:20:09.435592303 +0000 UTC m=+626.733732788" watchObservedRunningTime="2026-08-04 03:20:09.436764386 +0000 UTC m=+626.734904874"
Aug 04 03:20:09.700567 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:09.700471    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:09.800970 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:09.800898    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:11.379410 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:11.379383    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fg85w"
Aug 04 03:20:11.381185 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:11.381156    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:11.431142 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:11.431103    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f" event={"ID":"673df7a3-2a15-4067-af62-0ca57549b671","Type":"ContainerStarted","Data":"b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419"}
Aug 04 03:20:11.431311 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:11.431205    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:11.447890 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:11.447833    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f" podStartSLOduration=2.219513965 podStartE2EDuration="4.447818475s" totalImagesPullingTime="2.22830451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:20:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:20:11.446099698 +0000 UTC m=+628.744240187" watchObservedRunningTime="2026-08-04 03:20:11.447818475 +0000 UTC m=+628.745958989"
Aug 04 03:20:11.852687 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:11.852644    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:13.320754 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:13.320722    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:13.383498 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:13.383459    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:14.961525 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:14.961490    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:15.062670 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:15.062637    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:18.413325 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:18.413299    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4fmw6"
Aug 04 03:20:20.427641 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:20.427609    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:20.550499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:20.550469    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:20.652772 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:20.652735    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:21.899649 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:21.899524    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"]
Aug 04 03:20:21.901921 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:21.901904    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:21.914600 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:21.914573    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"]
Aug 04 03:20:21.988033 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:21.987999    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djn6f\" (UniqueName: \"kubernetes.io/projected/6249ddf7-c5cf-4228-b227-aabf1c6329d6-kube-api-access-djn6f\") pod \"kuadrant-operator-controller-manager-55467d9f7-szlmg\" (UID: \"6249ddf7-c5cf-4228-b227-aabf1c6329d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:21.988216 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:21.988048    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6249ddf7-c5cf-4228-b227-aabf1c6329d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-szlmg\" (UID: \"6249ddf7-c5cf-4228-b227-aabf1c6329d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:22.020596 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.020527    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"]
Aug 04 03:20:22.021001 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.020972    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-djn6f], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg" podUID="6249ddf7-c5cf-4228-b227-aabf1c6329d6"
Aug 04 03:20:22.038516 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.038466    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"]
Aug 04 03:20:22.049375 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.049318    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"]
Aug 04 03:20:22.054897 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.054857    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"]
Aug 04 03:20:22.058117 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.058084    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:20:22.062890 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.062854    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"]
Aug 04 03:20:22.063174 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.063140    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" containerName="manager" containerID="cri-o://a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50" gracePeriod=2
Aug 04 03:20:22.071914 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.071873    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"]
Aug 04 03:20:22.073476 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.073452    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"]
Aug 04 03:20:22.089011 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.088980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djn6f\" (UniqueName: \"kubernetes.io/projected/6249ddf7-c5cf-4228-b227-aabf1c6329d6-kube-api-access-djn6f\") pod \"kuadrant-operator-controller-manager-55467d9f7-szlmg\" (UID: \"6249ddf7-c5cf-4228-b227-aabf1c6329d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:22.089130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.089089    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6249ddf7-c5cf-4228-b227-aabf1c6329d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-szlmg\" (UID: \"6249ddf7-c5cf-4228-b227-aabf1c6329d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:22.089508 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.089482    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6249ddf7-c5cf-4228-b227-aabf1c6329d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-szlmg\" (UID: \"6249ddf7-c5cf-4228-b227-aabf1c6329d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:22.092382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.092360    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"]
Aug 04 03:20:22.092864 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.092840    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" containerName="manager"
Aug 04 03:20:22.092864 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.092859    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" containerName="manager"
Aug 04 03:20:22.092962 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.092956    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" containerName="manager"
Aug 04 03:20:22.094644 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.094625    2559 projected.go:196] Error preparing data for projected volume kube-api-access-djn6f for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-szlmg" not found
Aug 04 03:20:22.094741 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.094691    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6249ddf7-c5cf-4228-b227-aabf1c6329d6-kube-api-access-djn6f podName:6249ddf7-c5cf-4228-b227-aabf1c6329d6 nodeName:}" failed. No retries permitted until 2026-08-04 03:20:22.594672547 +0000 UTC m=+639.892813028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-djn6f" (UniqueName: "kubernetes.io/projected/6249ddf7-c5cf-4228-b227-aabf1c6329d6-kube-api-access-djn6f") pod "kuadrant-operator-controller-manager-55467d9f7-szlmg" (UID: "6249ddf7-c5cf-4228-b227-aabf1c6329d6") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-szlmg" not found
Aug 04 03:20:22.095156 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.095139    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"]
Aug 04 03:20:22.095253 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.095237    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:22.095320 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.095302    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f" podUID="673df7a3-2a15-4067-af62-0ca57549b671" containerName="manager" containerID="cri-o://b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419" gracePeriod=2
Aug 04 03:20:22.099747 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.099726    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.102374 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.102348    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"]
Aug 04 03:20:22.111329 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.111304    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"]
Aug 04 03:20:22.116463 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.116439    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4"]
Aug 04 03:20:22.116812 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.116798    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="673df7a3-2a15-4067-af62-0ca57549b671" containerName="manager"
Aug 04 03:20:22.116863 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.116814    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="673df7a3-2a15-4067-af62-0ca57549b671" containerName="manager"
Aug 04 03:20:22.116907 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.116863    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="673df7a3-2a15-4067-af62-0ca57549b671" containerName="manager"
Aug 04 03:20:22.118633 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.118619    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4"
Aug 04 03:20:22.125927 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.125899    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="673df7a3-2a15-4067-af62-0ca57549b671" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.127733 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.127707    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:22.129513 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.129484    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:22.133816 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.133789    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4"]
Aug 04 03:20:22.151495 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.151433    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="673df7a3-2a15-4067-af62-0ca57549b671" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.189727 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.189696    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnhn8\" (UniqueName: \"kubernetes.io/projected/4ed94a51-6745-4d9b-8ecf-db3d7f8841f1-kube-api-access-cnhn8\") pod \"limitador-operator-controller-manager-859d7b78bd-r69f4\" (UID: \"4ed94a51-6745-4d9b-8ecf-db3d7f8841f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4"
Aug 04 03:20:22.189878 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.189733    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2k7z\" (UniqueName: \"kubernetes.io/projected/d658da66-34f8-45e6-808a-a4b946939e09-kube-api-access-w2k7z\") pod \"kuadrant-operator-controller-manager-86878c769f-4n4r4\" (UID: \"d658da66-34f8-45e6-808a-a4b946939e09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:20:22.189878 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.189856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b3b631c-e59b-4967-b405-b16159ddec47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8rnfv\" (UID: \"4b3b631c-e59b-4967-b405-b16159ddec47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:22.189948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.189885    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hfqn\" (UniqueName: \"kubernetes.io/projected/4b3b631c-e59b-4967-b405-b16159ddec47-kube-api-access-2hfqn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8rnfv\" (UID: \"4b3b631c-e59b-4967-b405-b16159ddec47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:22.189948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.189906    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d658da66-34f8-45e6-808a-a4b946939e09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4n4r4\" (UID: \"d658da66-34f8-45e6-808a-a4b946939e09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:20:22.291097 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.291068    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnhn8\" (UniqueName: \"kubernetes.io/projected/4ed94a51-6745-4d9b-8ecf-db3d7f8841f1-kube-api-access-cnhn8\") pod \"limitador-operator-controller-manager-859d7b78bd-r69f4\" (UID: \"4ed94a51-6745-4d9b-8ecf-db3d7f8841f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4"
Aug 04 03:20:22.291097 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.291099    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2k7z\" (UniqueName: \"kubernetes.io/projected/d658da66-34f8-45e6-808a-a4b946939e09-kube-api-access-w2k7z\") pod \"kuadrant-operator-controller-manager-86878c769f-4n4r4\" (UID: \"d658da66-34f8-45e6-808a-a4b946939e09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:20:22.291318 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.291167    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b3b631c-e59b-4967-b405-b16159ddec47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8rnfv\" (UID: \"4b3b631c-e59b-4967-b405-b16159ddec47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:22.291318 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.291189    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hfqn\" (UniqueName: \"kubernetes.io/projected/4b3b631c-e59b-4967-b405-b16159ddec47-kube-api-access-2hfqn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8rnfv\" (UID: \"4b3b631c-e59b-4967-b405-b16159ddec47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:22.291318 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.291209    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d658da66-34f8-45e6-808a-a4b946939e09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4n4r4\" (UID: \"d658da66-34f8-45e6-808a-a4b946939e09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:20:22.291648 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.291624    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d658da66-34f8-45e6-808a-a4b946939e09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4n4r4\" (UID: \"d658da66-34f8-45e6-808a-a4b946939e09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:20:22.291726 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.291660    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b3b631c-e59b-4967-b405-b16159ddec47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8rnfv\" (UID: \"4b3b631c-e59b-4967-b405-b16159ddec47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:22.300807 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.300772    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnhn8\" (UniqueName: \"kubernetes.io/projected/4ed94a51-6745-4d9b-8ecf-db3d7f8841f1-kube-api-access-cnhn8\") pod \"limitador-operator-controller-manager-859d7b78bd-r69f4\" (UID: \"4ed94a51-6745-4d9b-8ecf-db3d7f8841f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4"
Aug 04 03:20:22.301055 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.301034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2k7z\" (UniqueName: \"kubernetes.io/projected/d658da66-34f8-45e6-808a-a4b946939e09-kube-api-access-w2k7z\") pod \"kuadrant-operator-controller-manager-86878c769f-4n4r4\" (UID: \"d658da66-34f8-45e6-808a-a4b946939e09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:20:22.301169 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.301147    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hfqn\" (UniqueName: \"kubernetes.io/projected/4b3b631c-e59b-4967-b405-b16159ddec47-kube-api-access-2hfqn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8rnfv\" (UID: \"4b3b631c-e59b-4967-b405-b16159ddec47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:22.332437 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.332414    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:22.334273 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.334244    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="673df7a3-2a15-4067-af62-0ca57549b671" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.334781 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.334764    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.336102 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.336073    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:22.337885 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.337865    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="673df7a3-2a15-4067-af62-0ca57549b671" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.339613 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.339589    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:22.376808 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.376776    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:20:22.391572 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.391547    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d101a74d-4c2b-446b-85b4-5c47ddf22d28-extensions-socket-volume\") pod \"d101a74d-4c2b-446b-85b4-5c47ddf22d28\" (UID: \"d101a74d-4c2b-446b-85b4-5c47ddf22d28\") "
Aug 04 03:20:22.391682 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.391587    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82kv4\" (UniqueName: \"kubernetes.io/projected/d101a74d-4c2b-446b-85b4-5c47ddf22d28-kube-api-access-82kv4\") pod \"d101a74d-4c2b-446b-85b4-5c47ddf22d28\" (UID: \"d101a74d-4c2b-446b-85b4-5c47ddf22d28\") "
Aug 04 03:20:22.392078 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.392048    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d101a74d-4c2b-446b-85b4-5c47ddf22d28-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d101a74d-4c2b-446b-85b4-5c47ddf22d28" (UID: "d101a74d-4c2b-446b-85b4-5c47ddf22d28"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:20:22.393480 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.393458    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d101a74d-4c2b-446b-85b4-5c47ddf22d28-kube-api-access-82kv4" (OuterVolumeSpecName: "kube-api-access-82kv4") pod "d101a74d-4c2b-446b-85b4-5c47ddf22d28" (UID: "d101a74d-4c2b-446b-85b4-5c47ddf22d28"). InnerVolumeSpecName "kube-api-access-82kv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:20:22.491479 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.491442    2559 generic.go:356] "Generic (PLEG): container finished" podID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" containerID="a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50" exitCode=0
Aug 04 03:20:22.491697 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.491485    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:22.491697 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.491554    2559 scope.go:117] "RemoveContainer" containerID="a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50"
Aug 04 03:20:22.492166 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.492131    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2mbw\" (UniqueName: \"kubernetes.io/projected/673df7a3-2a15-4067-af62-0ca57549b671-kube-api-access-g2mbw\") pod \"673df7a3-2a15-4067-af62-0ca57549b671\" (UID: \"673df7a3-2a15-4067-af62-0ca57549b671\") "
Aug 04 03:20:22.492437 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.492414    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d101a74d-4c2b-446b-85b4-5c47ddf22d28-extensions-socket-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:20:22.492560 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.492442    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82kv4\" (UniqueName: \"kubernetes.io/projected/d101a74d-4c2b-446b-85b4-5c47ddf22d28-kube-api-access-82kv4\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:20:22.494162 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.494130    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="673df7a3-2a15-4067-af62-0ca57549b671" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.494801 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.494769    2559 generic.go:356] "Generic (PLEG): container finished" podID="673df7a3-2a15-4067-af62-0ca57549b671" containerID="b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419" exitCode=0
Aug 04 03:20:22.494945 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.494865    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:22.494945 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.494930    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.495099 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.495070    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/673df7a3-2a15-4067-af62-0ca57549b671-kube-api-access-g2mbw" (OuterVolumeSpecName: "kube-api-access-g2mbw") pod "673df7a3-2a15-4067-af62-0ca57549b671" (UID: "673df7a3-2a15-4067-af62-0ca57549b671"). InnerVolumeSpecName "kube-api-access-g2mbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:20:22.496439 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.496403    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:22.498397 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.498369    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="673df7a3-2a15-4067-af62-0ca57549b671" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.500336 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.500309    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:22.503725 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.503701    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-szlmg\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="6249ddf7-c5cf-4228-b227-aabf1c6329d6" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:22.506140 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.506119    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:22.506587 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.506571    2559 scope.go:117] "RemoveContainer" containerID="a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50"
Aug 04 03:20:22.506868 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.506845    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="673df7a3-2a15-4067-af62-0ca57549b671" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.506950 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.506848    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50\": container with ID starting with a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50 not found: ID does not exist" containerID="a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50"
Aug 04 03:20:22.506991 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.506970    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50"} err="failed to get container status \"a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50\": rpc error: code = NotFound desc = could not find container \"a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50\": container with ID starting with a7e29a6c3a014738d7cd047dab76b9b64b5286f19343511c2195bbf3eb3f6c50 not found: ID does not exist"
Aug 04 03:20:22.507028 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.506995    2559 scope.go:117] "RemoveContainer" containerID="b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419"
Aug 04 03:20:22.508735 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.508710    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:22.510524 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.510502    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-szlmg\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="6249ddf7-c5cf-4228-b227-aabf1c6329d6" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:22.511640 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.511619    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:22.512493 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.512472    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="673df7a3-2a15-4067-af62-0ca57549b671" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.514373 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.514345    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:22.514929 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.514883    2559 scope.go:117] "RemoveContainer" containerID="b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419"
Aug 04 03:20:22.515181 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.515165    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419\": container with ID starting with b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419 not found: ID does not exist" containerID="b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419"
Aug 04 03:20:22.515247 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.515193    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419"} err="failed to get container status \"b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419\": rpc error: code = NotFound desc = could not find container \"b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419\": container with ID starting with b08afcd3ad77b9d53531432021e091b2ccb5a48feb1d4e6ea3f457ea75e4b419 not found: ID does not exist"
Aug 04 03:20:22.516255 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.516234    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-szlmg\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="6249ddf7-c5cf-4228-b227-aabf1c6329d6" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:22.517339 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.517324    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4"
Aug 04 03:20:22.555716 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.550609    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"]
Aug 04 03:20:22.592776 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.592713    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6249ddf7-c5cf-4228-b227-aabf1c6329d6-extensions-socket-volume\") pod \"6249ddf7-c5cf-4228-b227-aabf1c6329d6\" (UID: \"6249ddf7-c5cf-4228-b227-aabf1c6329d6\") "
Aug 04 03:20:22.593030 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.593012    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2mbw\" (UniqueName: \"kubernetes.io/projected/673df7a3-2a15-4067-af62-0ca57549b671-kube-api-access-g2mbw\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:20:22.593095 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.593038    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djn6f\" (UniqueName: \"kubernetes.io/projected/6249ddf7-c5cf-4228-b227-aabf1c6329d6-kube-api-access-djn6f\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:20:22.593095 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.593053    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6249ddf7-c5cf-4228-b227-aabf1c6329d6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6249ddf7-c5cf-4228-b227-aabf1c6329d6" (UID: "6249ddf7-c5cf-4228-b227-aabf1c6329d6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:20:22.666174 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.666149    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"]
Aug 04 03:20:22.668799 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:20:22.668770    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b3b631c_e59b_4967_b405_b16159ddec47.slice/crio-de4743e6e49dcea75ad4c05e957d7e6ac93c08c18a3dd97db0923776717779e5 WatchSource:0}: Error finding container de4743e6e49dcea75ad4c05e957d7e6ac93c08c18a3dd97db0923776717779e5: Status 404 returned error can't find the container with id de4743e6e49dcea75ad4c05e957d7e6ac93c08c18a3dd97db0923776717779e5
Aug 04 03:20:22.675096 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.675078    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4"]
Aug 04 03:20:22.677281 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:20:22.677257    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed94a51_6745_4d9b_8ecf_db3d7f8841f1.slice/crio-45c4e79b8855af8077198c526266d715895abae47f058a1abd4affe1c54699d2 WatchSource:0}: Error finding container 45c4e79b8855af8077198c526266d715895abae47f058a1abd4affe1c54699d2: Status 404 returned error can't find the container with id 45c4e79b8855af8077198c526266d715895abae47f058a1abd4affe1c54699d2
Aug 04 03:20:22.693738 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:22.693715    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6249ddf7-c5cf-4228-b227-aabf1c6329d6-extensions-socket-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:20:22.804890 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.804862    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="673df7a3-2a15-4067-af62-0ca57549b671" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:22.806496 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.806475    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:22.807976 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:22.807959    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-szlmg\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="6249ddf7-c5cf-4228-b227-aabf1c6329d6" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:23.235990 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:23.235947    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ghg5f\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="673df7a3-2a15-4067-af62-0ca57549b671" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ghg5f"
Aug 04 03:20:23.238761 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.238726    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6249ddf7-c5cf-4228-b227-aabf1c6329d6" path="/var/lib/kubelet/pods/6249ddf7-c5cf-4228-b227-aabf1c6329d6/volumes"
Aug 04 03:20:23.238919 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:23.238898    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-n7cs7\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7cs7"
Aug 04 03:20:23.238987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.238974    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="673df7a3-2a15-4067-af62-0ca57549b671" path="/var/lib/kubelet/pods/673df7a3-2a15-4067-af62-0ca57549b671/volumes"
Aug 04 03:20:23.239295 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.239284    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d101a74d-4c2b-446b-85b4-5c47ddf22d28" path="/var/lib/kubelet/pods/d101a74d-4c2b-446b-85b4-5c47ddf22d28/volumes"
Aug 04 03:20:23.240422 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:23.240393    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-szlmg\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="6249ddf7-c5cf-4228-b227-aabf1c6329d6" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:23.501289 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.501260    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4" event={"ID":"4ed94a51-6745-4d9b-8ecf-db3d7f8841f1","Type":"ContainerStarted","Data":"73f045ade3d03cb4c15c3b85b80077c63989af6cd5d3d2066460585048ec8e76"}
Aug 04 03:20:23.501497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.501313    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4" event={"ID":"4ed94a51-6745-4d9b-8ecf-db3d7f8841f1","Type":"ContainerStarted","Data":"45c4e79b8855af8077198c526266d715895abae47f058a1abd4affe1c54699d2"}
Aug 04 03:20:23.501497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.501342    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4"
Aug 04 03:20:23.503109 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.503085    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv" event={"ID":"4b3b631c-e59b-4967-b405-b16159ddec47","Type":"ContainerStarted","Data":"fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f"}
Aug 04 03:20:23.503219 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.503132    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv" event={"ID":"4b3b631c-e59b-4967-b405-b16159ddec47","Type":"ContainerStarted","Data":"de4743e6e49dcea75ad4c05e957d7e6ac93c08c18a3dd97db0923776717779e5"}
Aug 04 03:20:23.503219 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.503204    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:23.505610 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.505591    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4" event={"ID":"d658da66-34f8-45e6-808a-a4b946939e09","Type":"ContainerStarted","Data":"ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30"}
Aug 04 03:20:23.505712 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.505624    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4" event={"ID":"d658da66-34f8-45e6-808a-a4b946939e09","Type":"ContainerStarted","Data":"78591c31c175af88886b8c4b64814b8c8aa191374bdb80787166e6cc6bbddc9a"}
Aug 04 03:20:23.505712 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.505660    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:20:23.506531 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.506513    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:23.517410 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.517370    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4" podStartSLOduration=1.517355542 podStartE2EDuration="1.517355542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:20:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:20:23.516294039 +0000 UTC m=+640.814434529" watchObservedRunningTime="2026-08-04 03:20:23.517355542 +0000 UTC m=+640.815496021"
Aug 04 03:20:23.533506 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.533457    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4" podStartSLOduration=1.5334423099999999 podStartE2EDuration="1.53344231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:20:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:20:23.532084439 +0000 UTC m=+640.830224953" watchObservedRunningTime="2026-08-04 03:20:23.53344231 +0000 UTC m=+640.831582797"
Aug 04 03:20:23.549463 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:23.549408    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv" podStartSLOduration=1.549391073 podStartE2EDuration="1.549391073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:20:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:20:23.547793485 +0000 UTC m=+640.845933994" watchObservedRunningTime="2026-08-04 03:20:23.549391073 +0000 UTC m=+640.847531563"
Aug 04 03:20:23.549671 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:23.549528    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-szlmg\" is forbidden: User \"system:node:ip-10-0-134-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-110.ec2.internal' and this object" podUID="6249ddf7-c5cf-4228-b227-aabf1c6329d6" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-szlmg"
Aug 04 03:20:24.553091 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:24.553058    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:24.666152 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:24.666118    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:28.335390 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:28.335356    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:28.386982 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:28.386947    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:28.531826 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:28.531787    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:28.594154 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:28.594075    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:30.979521 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:30.979490    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:31.080495 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:31.080466    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:34.514007 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:34.513978    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:34.514481 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:34.514042    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r69f4"
Aug 04 03:20:34.514481 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:34.514073    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:20:34.612254 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:34.612219    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"]
Aug 04 03:20:34.612463 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:34.612415    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv" podUID="4b3b631c-e59b-4967-b405-b16159ddec47" containerName="manager" containerID="cri-o://fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f" gracePeriod=10
Aug 04 03:20:34.854848 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:34.854821    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:35.001714 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.001679    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hfqn\" (UniqueName: \"kubernetes.io/projected/4b3b631c-e59b-4967-b405-b16159ddec47-kube-api-access-2hfqn\") pod \"4b3b631c-e59b-4967-b405-b16159ddec47\" (UID: \"4b3b631c-e59b-4967-b405-b16159ddec47\") "
Aug 04 03:20:35.001904 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.001724    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b3b631c-e59b-4967-b405-b16159ddec47-extensions-socket-volume\") pod \"4b3b631c-e59b-4967-b405-b16159ddec47\" (UID: \"4b3b631c-e59b-4967-b405-b16159ddec47\") "
Aug 04 03:20:35.002137 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.002113    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b3b631c-e59b-4967-b405-b16159ddec47-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4b3b631c-e59b-4967-b405-b16159ddec47" (UID: "4b3b631c-e59b-4967-b405-b16159ddec47"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:20:35.003767 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.003741    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b3b631c-e59b-4967-b405-b16159ddec47-kube-api-access-2hfqn" (OuterVolumeSpecName: "kube-api-access-2hfqn") pod "4b3b631c-e59b-4967-b405-b16159ddec47" (UID: "4b3b631c-e59b-4967-b405-b16159ddec47"). InnerVolumeSpecName "kube-api-access-2hfqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:20:35.103126 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.103044    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hfqn\" (UniqueName: \"kubernetes.io/projected/4b3b631c-e59b-4967-b405-b16159ddec47-kube-api-access-2hfqn\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:20:35.103126 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.103075    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b3b631c-e59b-4967-b405-b16159ddec47-extensions-socket-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:20:35.565167 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.565132    2559 generic.go:356] "Generic (PLEG): container finished" podID="4b3b631c-e59b-4967-b405-b16159ddec47" containerID="fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f" exitCode=0
Aug 04 03:20:35.565167 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.565182    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"
Aug 04 03:20:35.565739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.565208    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv" event={"ID":"4b3b631c-e59b-4967-b405-b16159ddec47","Type":"ContainerDied","Data":"fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f"}
Aug 04 03:20:35.565739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.565265    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv" event={"ID":"4b3b631c-e59b-4967-b405-b16159ddec47","Type":"ContainerDied","Data":"de4743e6e49dcea75ad4c05e957d7e6ac93c08c18a3dd97db0923776717779e5"}
Aug 04 03:20:35.565739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.565288    2559 scope.go:117] "RemoveContainer" containerID="fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f"
Aug 04 03:20:35.573780 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.573760    2559 scope.go:117] "RemoveContainer" containerID="fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f"
Aug 04 03:20:35.574108 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:20:35.574089    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f\": container with ID starting with fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f not found: ID does not exist" containerID="fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f"
Aug 04 03:20:35.574163 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.574121    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f"} err="failed to get container status \"fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f\": rpc error: code = NotFound desc = could not find container \"fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f\": container with ID starting with fd927a571e4d20fc1f887b96b7a1796cc13118b81641fdb9f7cfcef4e858205f not found: ID does not exist"
Aug 04 03:20:35.584199 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.584169    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"]
Aug 04 03:20:35.586638 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:35.586605    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8rnfv"]
Aug 04 03:20:37.240704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:37.238768    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b3b631c-e59b-4967-b405-b16159ddec47" path="/var/lib/kubelet/pods/4b3b631c-e59b-4967-b405-b16159ddec47/volumes"
Aug 04 03:20:37.479085 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:37.479050    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:37.592725 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:37.592641    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:42.235157 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:42.235124    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:42.338744 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:42.338700    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:43.224780 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:43.224748    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:46.065998 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:46.065964    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:46.164407 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:46.164373    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:46.431922 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:46.431833    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:46.533895 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:46.533855    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:50.432047 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.432001    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:50.533888 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.533856    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:50.897057 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.897027    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"]
Aug 04 03:20:50.897386 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.897369    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b3b631c-e59b-4967-b405-b16159ddec47" containerName="manager"
Aug 04 03:20:50.897386 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.897386    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b3b631c-e59b-4967-b405-b16159ddec47" containerName="manager"
Aug 04 03:20:50.897528 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.897454    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b3b631c-e59b-4967-b405-b16159ddec47" containerName="manager"
Aug 04 03:20:50.903651 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.903631    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:50.905783 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.905766    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tq66m\""
Aug 04 03:20:50.917939 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.917917    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"]
Aug 04 03:20:50.920658 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.920635    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:50.920841 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.920814    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f328e85-83a1-461f-913b-d9d77efb5bf9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:50.920951 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.920909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms7cs\" (UniqueName: \"kubernetes.io/projected/6f328e85-83a1-461f-913b-d9d77efb5bf9-kube-api-access-ms7cs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:50.920951 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.920946    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:50.921055 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.920974    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:50.921117 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.921071    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:50.921117 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.921104    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:50.921223 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.921138    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:50.921223 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.921165    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:50.921333 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:50.921253    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022014 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.021975    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022014 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022016    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022259 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022046    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022259 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022068    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022259 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022259 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f328e85-83a1-461f-913b-d9d77efb5bf9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022259 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022227    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms7cs\" (UniqueName: \"kubernetes.io/projected/6f328e85-83a1-461f-913b-d9d77efb5bf9-kube-api-access-ms7cs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022504 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022287    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022504 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022334    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022504 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022377    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022504 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022434    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022749 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022604    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.022749 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.022715    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.023058 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.023035    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f328e85-83a1-461f-913b-d9d77efb5bf9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.024297 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.024277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.024534 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.024520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.031202 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.031182    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f328e85-83a1-461f-913b-d9d77efb5bf9-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.031399 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.031311    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms7cs\" (UniqueName: \"kubernetes.io/projected/6f328e85-83a1-461f-913b-d9d77efb5bf9-kube-api-access-ms7cs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-cbzlz\" (UID: \"6f328e85-83a1-461f-913b-d9d77efb5bf9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.139837 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.139798    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:51.215200 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.215132    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:51.352003 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.351900    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"]
Aug 04 03:20:51.354212 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:20:51.354187    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f328e85_83a1_461f_913b_d9d77efb5bf9.slice/crio-f04c8f01b415b50d493aa3ff77a5458c3912b2a35a465dfadd210905d0e61a08 WatchSource:0}: Error finding container f04c8f01b415b50d493aa3ff77a5458c3912b2a35a465dfadd210905d0e61a08: Status 404 returned error can't find the container with id f04c8f01b415b50d493aa3ff77a5458c3912b2a35a465dfadd210905d0e61a08
Aug 04 03:20:51.356311 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.356276    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:20:51.356426 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.356344    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:20:51.356426 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.356372    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:20:51.640433 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.640396    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz" event={"ID":"6f328e85-83a1-461f-913b-d9d77efb5bf9","Type":"ContainerStarted","Data":"72d491f0b07e10ef456d0ac3a1fa3058763b0e63862157fed8255fa8bc61e47a"}
Aug 04 03:20:51.640903 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.640447    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz" event={"ID":"6f328e85-83a1-461f-913b-d9d77efb5bf9","Type":"ContainerStarted","Data":"f04c8f01b415b50d493aa3ff77a5458c3912b2a35a465dfadd210905d0e61a08"}
Aug 04 03:20:51.659226 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:51.659175    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz" podStartSLOduration=1.659160381 podStartE2EDuration="1.659160381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:20:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:20:51.657947886 +0000 UTC m=+668.956088375" watchObservedRunningTime="2026-08-04 03:20:51.659160381 +0000 UTC m=+668.957300869"
Aug 04 03:20:52.215716 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:52.215686    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:52.220107 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:52.220085    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:52.608853 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:52.608815    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:52.645212 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:52.645178    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:52.646092 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:52.646073    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:20:52.742991 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:52.742962    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:52.846766 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:52.846736    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:53.401605 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.401568    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-h9ftv"]
Aug 04 03:20:53.405429 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.405407    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h9ftv"
Aug 04 03:20:53.407795 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.407768    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xxg24\""
Aug 04 03:20:53.409087 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.409063    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-h9ftv"]
Aug 04 03:20:53.438111 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.438080    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:53.441430 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.441410    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgvfs\" (UniqueName: \"kubernetes.io/projected/98f4fe0a-ce98-4b87-814c-fec73ec2a08c-kube-api-access-pgvfs\") pod \"authorino-58cd955675-h9ftv\" (UID: \"98f4fe0a-ce98-4b87-814c-fec73ec2a08c\") " pod="kuadrant-system/authorino-58cd955675-h9ftv"
Aug 04 03:20:53.537934 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.537900    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:53.541913 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.541891    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgvfs\" (UniqueName: \"kubernetes.io/projected/98f4fe0a-ce98-4b87-814c-fec73ec2a08c-kube-api-access-pgvfs\") pod \"authorino-58cd955675-h9ftv\" (UID: \"98f4fe0a-ce98-4b87-814c-fec73ec2a08c\") " pod="kuadrant-system/authorino-58cd955675-h9ftv"
Aug 04 03:20:53.558976 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.558946    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgvfs\" (UniqueName: \"kubernetes.io/projected/98f4fe0a-ce98-4b87-814c-fec73ec2a08c-kube-api-access-pgvfs\") pod \"authorino-58cd955675-h9ftv\" (UID: \"98f4fe0a-ce98-4b87-814c-fec73ec2a08c\") " pod="kuadrant-system/authorino-58cd955675-h9ftv"
Aug 04 03:20:53.674469 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.674392    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-t869q"]
Aug 04 03:20:53.677874 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.677859    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t869q"
Aug 04 03:20:53.686685 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.686659    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t869q"]
Aug 04 03:20:53.715686 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.715657    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h9ftv"
Aug 04 03:20:53.744114 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.744054    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt44c\" (UniqueName: \"kubernetes.io/projected/57548ba8-9cd2-45a9-8c1d-4666fe2471a1-kube-api-access-qt44c\") pod \"authorino-75d556bd77-t869q\" (UID: \"57548ba8-9cd2-45a9-8c1d-4666fe2471a1\") " pod="kuadrant-system/authorino-75d556bd77-t869q"
Aug 04 03:20:53.837447 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.837421    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-h9ftv"]
Aug 04 03:20:53.840029 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:20:53.839995    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98f4fe0a_ce98_4b87_814c_fec73ec2a08c.slice/crio-8a229ddf5164e50f175f595ae4eb3e31c9bb3f9bc8d7c6b829382b02de1ee761 WatchSource:0}: Error finding container 8a229ddf5164e50f175f595ae4eb3e31c9bb3f9bc8d7c6b829382b02de1ee761: Status 404 returned error can't find the container with id 8a229ddf5164e50f175f595ae4eb3e31c9bb3f9bc8d7c6b829382b02de1ee761
Aug 04 03:20:53.845337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.845306    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt44c\" (UniqueName: \"kubernetes.io/projected/57548ba8-9cd2-45a9-8c1d-4666fe2471a1-kube-api-access-qt44c\") pod \"authorino-75d556bd77-t869q\" (UID: \"57548ba8-9cd2-45a9-8c1d-4666fe2471a1\") " pod="kuadrant-system/authorino-75d556bd77-t869q"
Aug 04 03:20:53.852288 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.852246    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt44c\" (UniqueName: \"kubernetes.io/projected/57548ba8-9cd2-45a9-8c1d-4666fe2471a1-kube-api-access-qt44c\") pod \"authorino-75d556bd77-t869q\" (UID: \"57548ba8-9cd2-45a9-8c1d-4666fe2471a1\") " pod="kuadrant-system/authorino-75d556bd77-t869q"
Aug 04 03:20:53.987781 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:53.987730    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t869q"
Aug 04 03:20:54.110059 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:54.110035    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t869q"]
Aug 04 03:20:54.111965 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:20:54.111939    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57548ba8_9cd2_45a9_8c1d_4666fe2471a1.slice/crio-0d5f2e4e4a8248e303df6a4692be997f9e86d75cf53b0a23e87b674e39ebfb0c WatchSource:0}: Error finding container 0d5f2e4e4a8248e303df6a4692be997f9e86d75cf53b0a23e87b674e39ebfb0c: Status 404 returned error can't find the container with id 0d5f2e4e4a8248e303df6a4692be997f9e86d75cf53b0a23e87b674e39ebfb0c
Aug 04 03:20:54.655900 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:54.655849    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t869q" event={"ID":"57548ba8-9cd2-45a9-8c1d-4666fe2471a1","Type":"ContainerStarted","Data":"0d5f2e4e4a8248e303df6a4692be997f9e86d75cf53b0a23e87b674e39ebfb0c"}
Aug 04 03:20:54.657350 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:54.657295    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h9ftv" event={"ID":"98f4fe0a-ce98-4b87-814c-fec73ec2a08c","Type":"ContainerStarted","Data":"8a229ddf5164e50f175f595ae4eb3e31c9bb3f9bc8d7c6b829382b02de1ee761"}
Aug 04 03:20:55.076515 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:55.076472    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:55.176151 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:55.176119    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:57.678973 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:57.678929    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t869q" event={"ID":"57548ba8-9cd2-45a9-8c1d-4666fe2471a1","Type":"ContainerStarted","Data":"8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7"}
Aug 04 03:20:57.680735 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:57.680707    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h9ftv" event={"ID":"98f4fe0a-ce98-4b87-814c-fec73ec2a08c","Type":"ContainerStarted","Data":"25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b"}
Aug 04 03:20:57.693383 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:57.693326    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-t869q" podStartSLOduration=1.258309555 podStartE2EDuration="4.693310597s" totalImagesPullingTime="3.435001042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:20:57.69195649 +0000 UTC m=+674.990096969" watchObservedRunningTime="2026-08-04 03:20:57.693310597 +0000 UTC m=+674.991451129"
Aug 04 03:20:57.705503 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:57.705457    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-h9ftv" podStartSLOduration=1.009487634 podStartE2EDuration="4.705444017s" totalImagesPullingTime="3.695956383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:20:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:20:57.704944397 +0000 UTC m=+675.003084887" watchObservedRunningTime="2026-08-04 03:20:57.705444017 +0000 UTC m=+675.003584506"
Aug 04 03:20:57.727670 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:57.727639    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-h9ftv"]
Aug 04 03:20:57.829871 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:57.829774    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:57.883649 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:57.883605    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:20:59.688457 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:59.688416    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-h9ftv" podUID="98f4fe0a-ce98-4b87-814c-fec73ec2a08c" containerName="authorino" containerID="cri-o://25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b" gracePeriod=30
Aug 04 03:20:59.922408 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:59.922383    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h9ftv"
Aug 04 03:20:59.995439 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:59.995409    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgvfs\" (UniqueName: \"kubernetes.io/projected/98f4fe0a-ce98-4b87-814c-fec73ec2a08c-kube-api-access-pgvfs\") pod \"98f4fe0a-ce98-4b87-814c-fec73ec2a08c\" (UID: \"98f4fe0a-ce98-4b87-814c-fec73ec2a08c\") "
Aug 04 03:20:59.997426 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:20:59.997394    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f4fe0a-ce98-4b87-814c-fec73ec2a08c-kube-api-access-pgvfs" (OuterVolumeSpecName: "kube-api-access-pgvfs") pod "98f4fe0a-ce98-4b87-814c-fec73ec2a08c" (UID: "98f4fe0a-ce98-4b87-814c-fec73ec2a08c"). InnerVolumeSpecName "kube-api-access-pgvfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:21:00.095989 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:00.095961    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgvfs\" (UniqueName: \"kubernetes.io/projected/98f4fe0a-ce98-4b87-814c-fec73ec2a08c-kube-api-access-pgvfs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:21:00.694098 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:00.694062    2559 generic.go:356] "Generic (PLEG): container finished" podID="98f4fe0a-ce98-4b87-814c-fec73ec2a08c" containerID="25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b" exitCode=0
Aug 04 03:21:00.694098 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:00.694101    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-h9ftv"
Aug 04 03:21:00.694575 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:00.694111    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h9ftv" event={"ID":"98f4fe0a-ce98-4b87-814c-fec73ec2a08c","Type":"ContainerDied","Data":"25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b"}
Aug 04 03:21:00.694575 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:00.694148    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-h9ftv" event={"ID":"98f4fe0a-ce98-4b87-814c-fec73ec2a08c","Type":"ContainerDied","Data":"8a229ddf5164e50f175f595ae4eb3e31c9bb3f9bc8d7c6b829382b02de1ee761"}
Aug 04 03:21:00.694575 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:00.694166    2559 scope.go:117] "RemoveContainer" containerID="25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b"
Aug 04 03:21:00.702838 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:00.702822    2559 scope.go:117] "RemoveContainer" containerID="25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b"
Aug 04 03:21:00.703125 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:21:00.703102    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b\": container with ID starting with 25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b not found: ID does not exist" containerID="25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b"
Aug 04 03:21:00.703197 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:00.703132    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b"} err="failed to get container status \"25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b\": rpc error: code = NotFound desc = could not find container \"25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b\": container with ID starting with 25584a94d5fa54debd56d4483b2a75d9dc349369ddaf03957a6d4aa48b0ad31b not found: ID does not exist"
Aug 04 03:21:00.713671 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:00.713648    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-h9ftv"]
Aug 04 03:21:00.716899 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:00.716875    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-h9ftv"]
Aug 04 03:21:01.237583 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:01.237534    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98f4fe0a-ce98-4b87-814c-fec73ec2a08c" path="/var/lib/kubelet/pods/98f4fe0a-ce98-4b87-814c-fec73ec2a08c/volumes"
Aug 04 03:21:01.640979 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:01.640903    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:01.741627 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:01.741597    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:08.677794 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:08.677759    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:08.777292 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:08.777254    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:09.140631 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:09.140585    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:20.955852 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:20.955817    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:21.174369 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.174333    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7gdqn"]
Aug 04 03:21:21.174725 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.174711    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f4fe0a-ce98-4b87-814c-fec73ec2a08c" containerName="authorino"
Aug 04 03:21:21.174772 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.174727    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f4fe0a-ce98-4b87-814c-fec73ec2a08c" containerName="authorino"
Aug 04 03:21:21.174822 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.174811    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f4fe0a-ce98-4b87-814c-fec73ec2a08c" containerName="authorino"
Aug 04 03:21:21.177647 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.177631    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7gdqn"
Aug 04 03:21:21.183210 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.183186    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7gdqn"]
Aug 04 03:21:21.276970 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.276939    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpxmp\" (UniqueName: \"kubernetes.io/projected/734f6ed5-9df3-4734-9b5f-d8a3cd75d221-kube-api-access-lpxmp\") pod \"authorino-5d4b67c446-7gdqn\" (UID: \"734f6ed5-9df3-4734-9b5f-d8a3cd75d221\") " pod="kuadrant-system/authorino-5d4b67c446-7gdqn"
Aug 04 03:21:21.377489 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.377453    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpxmp\" (UniqueName: \"kubernetes.io/projected/734f6ed5-9df3-4734-9b5f-d8a3cd75d221-kube-api-access-lpxmp\") pod \"authorino-5d4b67c446-7gdqn\" (UID: \"734f6ed5-9df3-4734-9b5f-d8a3cd75d221\") " pod="kuadrant-system/authorino-5d4b67c446-7gdqn"
Aug 04 03:21:21.391659 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.391623    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpxmp\" (UniqueName: \"kubernetes.io/projected/734f6ed5-9df3-4734-9b5f-d8a3cd75d221-kube-api-access-lpxmp\") pod \"authorino-5d4b67c446-7gdqn\" (UID: \"734f6ed5-9df3-4734-9b5f-d8a3cd75d221\") " pod="kuadrant-system/authorino-5d4b67c446-7gdqn"
Aug 04 03:21:21.395799 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.395773    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7gdqn"]
Aug 04 03:21:21.396026 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.396012    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7gdqn"
Aug 04 03:21:21.422579 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.422552    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5458495cdb-f54qs"]
Aug 04 03:21:21.426879 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.426855    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5458495cdb-f54qs"
Aug 04 03:21:21.432607 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.432586    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5458495cdb-f54qs"]
Aug 04 03:21:21.478949 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.478915    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz22x\" (UniqueName: \"kubernetes.io/projected/290e48dc-8fcb-4e3e-ac64-b428d48473fe-kube-api-access-sz22x\") pod \"authorino-5458495cdb-f54qs\" (UID: \"290e48dc-8fcb-4e3e-ac64-b428d48473fe\") " pod="kuadrant-system/authorino-5458495cdb-f54qs"
Aug 04 03:21:21.513874 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.513834    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:21.529402 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.529339    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7gdqn"]
Aug 04 03:21:21.532353 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:21:21.532329    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod734f6ed5_9df3_4734_9b5f_d8a3cd75d221.slice/crio-b5c54c3184f394e7ee64809d5af46aae7214db687f3ccae23d44f0a1c6969905 WatchSource:0}: Error finding container b5c54c3184f394e7ee64809d5af46aae7214db687f3ccae23d44f0a1c6969905: Status 404 returned error can't find the container with id b5c54c3184f394e7ee64809d5af46aae7214db687f3ccae23d44f0a1c6969905
Aug 04 03:21:21.533623 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.533606    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 03:21:21.580039 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.580011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz22x\" (UniqueName: \"kubernetes.io/projected/290e48dc-8fcb-4e3e-ac64-b428d48473fe-kube-api-access-sz22x\") pod \"authorino-5458495cdb-f54qs\" (UID: \"290e48dc-8fcb-4e3e-ac64-b428d48473fe\") " pod="kuadrant-system/authorino-5458495cdb-f54qs"
Aug 04 03:21:21.587323 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.587302    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz22x\" (UniqueName: \"kubernetes.io/projected/290e48dc-8fcb-4e3e-ac64-b428d48473fe-kube-api-access-sz22x\") pod \"authorino-5458495cdb-f54qs\" (UID: \"290e48dc-8fcb-4e3e-ac64-b428d48473fe\") " pod="kuadrant-system/authorino-5458495cdb-f54qs"
Aug 04 03:21:21.663783 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.663753    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5458495cdb-f54qs"]
Aug 04 03:21:21.663996 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.663985    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5458495cdb-f54qs"
Aug 04 03:21:21.688667 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.688632    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b55cb4b98-lvtt6"]
Aug 04 03:21:21.693125 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.693102    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b55cb4b98-lvtt6"
Aug 04 03:21:21.695235 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.695214    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Aug 04 03:21:21.698805 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.698781    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b55cb4b98-lvtt6"]
Aug 04 03:21:21.782038 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.781959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58w9j\" (UniqueName: \"kubernetes.io/projected/8ce70c95-6011-461e-af2e-61b8c202065a-kube-api-access-58w9j\") pod \"authorino-7b55cb4b98-lvtt6\" (UID: \"8ce70c95-6011-461e-af2e-61b8c202065a\") " pod="kuadrant-system/authorino-7b55cb4b98-lvtt6"
Aug 04 03:21:21.782186 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.782058    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ce70c95-6011-461e-af2e-61b8c202065a-tls-cert\") pod \"authorino-7b55cb4b98-lvtt6\" (UID: \"8ce70c95-6011-461e-af2e-61b8c202065a\") " pod="kuadrant-system/authorino-7b55cb4b98-lvtt6"
Aug 04 03:21:21.788502 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.788476    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7gdqn" event={"ID":"734f6ed5-9df3-4734-9b5f-d8a3cd75d221","Type":"ContainerStarted","Data":"b5c54c3184f394e7ee64809d5af46aae7214db687f3ccae23d44f0a1c6969905"}
Aug 04 03:21:21.791376 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.791350    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5458495cdb-f54qs"]
Aug 04 03:21:21.793240 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:21:21.793217    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod290e48dc_8fcb_4e3e_ac64_b428d48473fe.slice/crio-a2c6607a5635191bda38a29f74e54e390c6b4595d03c0cb4bed7831269a226f3 WatchSource:0}: Error finding container a2c6607a5635191bda38a29f74e54e390c6b4595d03c0cb4bed7831269a226f3: Status 404 returned error can't find the container with id a2c6607a5635191bda38a29f74e54e390c6b4595d03c0cb4bed7831269a226f3
Aug 04 03:21:21.883210 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.883187    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58w9j\" (UniqueName: \"kubernetes.io/projected/8ce70c95-6011-461e-af2e-61b8c202065a-kube-api-access-58w9j\") pod \"authorino-7b55cb4b98-lvtt6\" (UID: \"8ce70c95-6011-461e-af2e-61b8c202065a\") " pod="kuadrant-system/authorino-7b55cb4b98-lvtt6"
Aug 04 03:21:21.883307 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.883231    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ce70c95-6011-461e-af2e-61b8c202065a-tls-cert\") pod \"authorino-7b55cb4b98-lvtt6\" (UID: \"8ce70c95-6011-461e-af2e-61b8c202065a\") " pod="kuadrant-system/authorino-7b55cb4b98-lvtt6"
Aug 04 03:21:21.885844 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.885823    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ce70c95-6011-461e-af2e-61b8c202065a-tls-cert\") pod \"authorino-7b55cb4b98-lvtt6\" (UID: \"8ce70c95-6011-461e-af2e-61b8c202065a\") " pod="kuadrant-system/authorino-7b55cb4b98-lvtt6"
Aug 04 03:21:21.892369 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:21.892348    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58w9j\" (UniqueName: \"kubernetes.io/projected/8ce70c95-6011-461e-af2e-61b8c202065a-kube-api-access-58w9j\") pod \"authorino-7b55cb4b98-lvtt6\" (UID: \"8ce70c95-6011-461e-af2e-61b8c202065a\") " pod="kuadrant-system/authorino-7b55cb4b98-lvtt6"
Aug 04 03:21:22.003867 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.003821    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b55cb4b98-lvtt6"
Aug 04 03:21:22.135597 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.135477    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b55cb4b98-lvtt6"]
Aug 04 03:21:22.138206 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:21:22.138169    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ce70c95_6011_461e_af2e_61b8c202065a.slice/crio-cb01146500f9d50fedb8649cf2f92934eed6524cccb2ecb108a344e4b5da2833 WatchSource:0}: Error finding container cb01146500f9d50fedb8649cf2f92934eed6524cccb2ecb108a344e4b5da2833: Status 404 returned error can't find the container with id cb01146500f9d50fedb8649cf2f92934eed6524cccb2ecb108a344e4b5da2833
Aug 04 03:21:22.794697 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.794659    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b55cb4b98-lvtt6" event={"ID":"8ce70c95-6011-461e-af2e-61b8c202065a","Type":"ContainerStarted","Data":"1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a"}
Aug 04 03:21:22.794890 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.794710    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b55cb4b98-lvtt6" event={"ID":"8ce70c95-6011-461e-af2e-61b8c202065a","Type":"ContainerStarted","Data":"cb01146500f9d50fedb8649cf2f92934eed6524cccb2ecb108a344e4b5da2833"}
Aug 04 03:21:22.796141 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.796109    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5458495cdb-f54qs" event={"ID":"290e48dc-8fcb-4e3e-ac64-b428d48473fe","Type":"ContainerStarted","Data":"12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c"}
Aug 04 03:21:22.796141 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.796126    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5458495cdb-f54qs" podUID="290e48dc-8fcb-4e3e-ac64-b428d48473fe" containerName="authorino" containerID="cri-o://12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c" gracePeriod=30
Aug 04 03:21:22.796348 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.796142    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5458495cdb-f54qs" event={"ID":"290e48dc-8fcb-4e3e-ac64-b428d48473fe","Type":"ContainerStarted","Data":"a2c6607a5635191bda38a29f74e54e390c6b4595d03c0cb4bed7831269a226f3"}
Aug 04 03:21:22.797723 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.797699    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7gdqn" event={"ID":"734f6ed5-9df3-4734-9b5f-d8a3cd75d221","Type":"ContainerStarted","Data":"cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e"}
Aug 04 03:21:22.797832 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.797778    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-7gdqn" podUID="734f6ed5-9df3-4734-9b5f-d8a3cd75d221" containerName="authorino" containerID="cri-o://cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e" gracePeriod=30
Aug 04 03:21:22.811638 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.811580    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b55cb4b98-lvtt6" podStartSLOduration=1.476025982 podStartE2EDuration="1.811564996s" totalImagesPullingTime="335.539014ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:21:22.80960238 +0000 UTC m=+700.107742960" watchObservedRunningTime="2026-08-04 03:21:22.811564996 +0000 UTC m=+700.109705484"
Aug 04 03:21:22.825100 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.825044    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-7gdqn" podStartSLOduration=1.477757945 podStartE2EDuration="1.825022803s" totalImagesPullingTime="347.264858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:21:22.823058334 +0000 UTC m=+700.121198829" watchObservedRunningTime="2026-08-04 03:21:22.825022803 +0000 UTC m=+700.123163291"
Aug 04 03:21:22.837024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.836973    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5458495cdb-f54qs" podStartSLOduration=1.403906836 podStartE2EDuration="1.83695912s" totalImagesPullingTime="433.052284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:21:22.835831581 +0000 UTC m=+700.133972069" watchObservedRunningTime="2026-08-04 03:21:22.83695912 +0000 UTC m=+700.135099608"
Aug 04 03:21:22.848900 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.848873    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t869q"]
Aug 04 03:21:22.849065 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:22.849045    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-t869q" podUID="57548ba8-9cd2-45a9-8c1d-4666fe2471a1" containerName="authorino" containerID="cri-o://8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7" gracePeriod=30
Aug 04 03:21:23.118863 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.118833    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7gdqn"
Aug 04 03:21:23.128647 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.128615    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5458495cdb-f54qs"
Aug 04 03:21:23.131619 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.131594    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t869q"
Aug 04 03:21:23.196848 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.196810    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qt44c\" (UniqueName: \"kubernetes.io/projected/57548ba8-9cd2-45a9-8c1d-4666fe2471a1-kube-api-access-qt44c\") pod \"57548ba8-9cd2-45a9-8c1d-4666fe2471a1\" (UID: \"57548ba8-9cd2-45a9-8c1d-4666fe2471a1\") "
Aug 04 03:21:23.196979 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.196884    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpxmp\" (UniqueName: \"kubernetes.io/projected/734f6ed5-9df3-4734-9b5f-d8a3cd75d221-kube-api-access-lpxmp\") pod \"734f6ed5-9df3-4734-9b5f-d8a3cd75d221\" (UID: \"734f6ed5-9df3-4734-9b5f-d8a3cd75d221\") "
Aug 04 03:21:23.196979 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.196938    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz22x\" (UniqueName: \"kubernetes.io/projected/290e48dc-8fcb-4e3e-ac64-b428d48473fe-kube-api-access-sz22x\") pod \"290e48dc-8fcb-4e3e-ac64-b428d48473fe\" (UID: \"290e48dc-8fcb-4e3e-ac64-b428d48473fe\") "
Aug 04 03:21:23.198941 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.198911    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/734f6ed5-9df3-4734-9b5f-d8a3cd75d221-kube-api-access-lpxmp" (OuterVolumeSpecName: "kube-api-access-lpxmp") pod "734f6ed5-9df3-4734-9b5f-d8a3cd75d221" (UID: "734f6ed5-9df3-4734-9b5f-d8a3cd75d221"). InnerVolumeSpecName "kube-api-access-lpxmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:21:23.199078 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.198956    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/290e48dc-8fcb-4e3e-ac64-b428d48473fe-kube-api-access-sz22x" (OuterVolumeSpecName: "kube-api-access-sz22x") pod "290e48dc-8fcb-4e3e-ac64-b428d48473fe" (UID: "290e48dc-8fcb-4e3e-ac64-b428d48473fe"). InnerVolumeSpecName "kube-api-access-sz22x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:21:23.199078 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.198974    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57548ba8-9cd2-45a9-8c1d-4666fe2471a1-kube-api-access-qt44c" (OuterVolumeSpecName: "kube-api-access-qt44c") pod "57548ba8-9cd2-45a9-8c1d-4666fe2471a1" (UID: "57548ba8-9cd2-45a9-8c1d-4666fe2471a1"). InnerVolumeSpecName "kube-api-access-qt44c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:21:23.297678 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.297650    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz22x\" (UniqueName: \"kubernetes.io/projected/290e48dc-8fcb-4e3e-ac64-b428d48473fe-kube-api-access-sz22x\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:21:23.297678 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.297677    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt44c\" (UniqueName: \"kubernetes.io/projected/57548ba8-9cd2-45a9-8c1d-4666fe2471a1-kube-api-access-qt44c\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:21:23.297871 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.297691    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpxmp\" (UniqueName: \"kubernetes.io/projected/734f6ed5-9df3-4734-9b5f-d8a3cd75d221-kube-api-access-lpxmp\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:21:23.803677 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.803648    2559 generic.go:356] "Generic (PLEG): container finished" podID="57548ba8-9cd2-45a9-8c1d-4666fe2471a1" containerID="8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7" exitCode=0
Aug 04 03:21:23.803875 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.803691    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t869q"
Aug 04 03:21:23.803875 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.803735    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t869q" event={"ID":"57548ba8-9cd2-45a9-8c1d-4666fe2471a1","Type":"ContainerDied","Data":"8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7"}
Aug 04 03:21:23.803875 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.803783    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t869q" event={"ID":"57548ba8-9cd2-45a9-8c1d-4666fe2471a1","Type":"ContainerDied","Data":"0d5f2e4e4a8248e303df6a4692be997f9e86d75cf53b0a23e87b674e39ebfb0c"}
Aug 04 03:21:23.803875 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.803804    2559 scope.go:117] "RemoveContainer" containerID="8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7"
Aug 04 03:21:23.805198 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.805029    2559 generic.go:356] "Generic (PLEG): container finished" podID="290e48dc-8fcb-4e3e-ac64-b428d48473fe" containerID="12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c" exitCode=0
Aug 04 03:21:23.805198 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.805066    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5458495cdb-f54qs"
Aug 04 03:21:23.805198 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.805127    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5458495cdb-f54qs" event={"ID":"290e48dc-8fcb-4e3e-ac64-b428d48473fe","Type":"ContainerDied","Data":"12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c"}
Aug 04 03:21:23.805198 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.805151    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5458495cdb-f54qs" event={"ID":"290e48dc-8fcb-4e3e-ac64-b428d48473fe","Type":"ContainerDied","Data":"a2c6607a5635191bda38a29f74e54e390c6b4595d03c0cb4bed7831269a226f3"}
Aug 04 03:21:23.806621 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.806595    2559 generic.go:356] "Generic (PLEG): container finished" podID="734f6ed5-9df3-4734-9b5f-d8a3cd75d221" containerID="cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e" exitCode=0
Aug 04 03:21:23.806743 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.806633    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7gdqn"
Aug 04 03:21:23.806743 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.806690    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7gdqn" event={"ID":"734f6ed5-9df3-4734-9b5f-d8a3cd75d221","Type":"ContainerDied","Data":"cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e"}
Aug 04 03:21:23.806743 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.806739    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7gdqn" event={"ID":"734f6ed5-9df3-4734-9b5f-d8a3cd75d221","Type":"ContainerDied","Data":"b5c54c3184f394e7ee64809d5af46aae7214db687f3ccae23d44f0a1c6969905"}
Aug 04 03:21:23.812996 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.812980    2559 scope.go:117] "RemoveContainer" containerID="8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7"
Aug 04 03:21:23.813262 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:21:23.813244    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7\": container with ID starting with 8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7 not found: ID does not exist" containerID="8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7"
Aug 04 03:21:23.813329 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.813270    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7"} err="failed to get container status \"8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7\": rpc error: code = NotFound desc = could not find container \"8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7\": container with ID starting with 8b9be59e34f15aac6433d16a11f01ebc5568cb1f8acda09a05cdbc9047c8f3f7 not found: ID does not exist"
Aug 04 03:21:23.813329 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.813287    2559 scope.go:117] "RemoveContainer" containerID="12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c"
Aug 04 03:21:23.820729 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.820712    2559 scope.go:117] "RemoveContainer" containerID="12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c"
Aug 04 03:21:23.820986 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:21:23.820968    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c\": container with ID starting with 12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c not found: ID does not exist" containerID="12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c"
Aug 04 03:21:23.821041 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.820995    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c"} err="failed to get container status \"12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c\": rpc error: code = NotFound desc = could not find container \"12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c\": container with ID starting with 12a7fa689487630d0234adb0005d46e43081bd75745edecd1d3ddcb8514e2b8c not found: ID does not exist"
Aug 04 03:21:23.821041 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.821013    2559 scope.go:117] "RemoveContainer" containerID="cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e"
Aug 04 03:21:23.826630 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.826609    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t869q"]
Aug 04 03:21:23.829257 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.829237    2559 scope.go:117] "RemoveContainer" containerID="cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e"
Aug 04 03:21:23.829663 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:21:23.829641    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e\": container with ID starting with cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e not found: ID does not exist" containerID="cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e"
Aug 04 03:21:23.829730 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.829670    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e"} err="failed to get container status \"cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e\": rpc error: code = NotFound desc = could not find container \"cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e\": container with ID starting with cba476bca7eba3bbbfb5e4910234470741dbd9347bfd6d7aa0d0884c6d157d3e not found: ID does not exist"
Aug 04 03:21:23.830037 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.830021    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t869q"]
Aug 04 03:21:23.838974 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.838951    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7gdqn"]
Aug 04 03:21:23.840717 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.840699    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7gdqn"]
Aug 04 03:21:23.850273 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.850251    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5458495cdb-f54qs"]
Aug 04 03:21:23.853559 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:23.853520    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5458495cdb-f54qs"]
Aug 04 03:21:25.219578 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:25.219524    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:25.237804 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:25.237776    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="290e48dc-8fcb-4e3e-ac64-b428d48473fe" path="/var/lib/kubelet/pods/290e48dc-8fcb-4e3e-ac64-b428d48473fe/volumes"
Aug 04 03:21:25.238081 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:25.238069    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57548ba8-9cd2-45a9-8c1d-4666fe2471a1" path="/var/lib/kubelet/pods/57548ba8-9cd2-45a9-8c1d-4666fe2471a1/volumes"
Aug 04 03:21:25.238336 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:25.238325    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="734f6ed5-9df3-4734-9b5f-d8a3cd75d221" path="/var/lib/kubelet/pods/734f6ed5-9df3-4734-9b5f-d8a3cd75d221/volumes"
Aug 04 03:21:27.259112 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:27.259075    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:27.322844 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:27.322808    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:27.939968 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:27.939929    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:31.024417 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:31.024369    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:21:31.120357 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:21:31.120318    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:02.636952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:02.636912    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:02.742728 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:02.742691    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:04.116978 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.116942    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:04.407658 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.407567    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:04.821368 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.821323    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:04.890653 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.890613    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:04.988455 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.988415    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"]
Aug 04 03:22:04.988906 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.988888    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="734f6ed5-9df3-4734-9b5f-d8a3cd75d221" containerName="authorino"
Aug 04 03:22:04.989003 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.988908    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="734f6ed5-9df3-4734-9b5f-d8a3cd75d221" containerName="authorino"
Aug 04 03:22:04.989003 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.988924    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57548ba8-9cd2-45a9-8c1d-4666fe2471a1" containerName="authorino"
Aug 04 03:22:04.989003 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.988932    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="57548ba8-9cd2-45a9-8c1d-4666fe2471a1" containerName="authorino"
Aug 04 03:22:04.989003 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.988947    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="290e48dc-8fcb-4e3e-ac64-b428d48473fe" containerName="authorino"
Aug 04 03:22:04.989003 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.988956    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="290e48dc-8fcb-4e3e-ac64-b428d48473fe" containerName="authorino"
Aug 04 03:22:04.989247 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.989037    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="734f6ed5-9df3-4734-9b5f-d8a3cd75d221" containerName="authorino"
Aug 04 03:22:04.989247 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.989051    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="290e48dc-8fcb-4e3e-ac64-b428d48473fe" containerName="authorino"
Aug 04 03:22:04.989247 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.989062    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="57548ba8-9cd2-45a9-8c1d-4666fe2471a1" containerName="authorino"
Aug 04 03:22:04.993594 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.993571    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"
Aug 04 03:22:04.995809 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.995786    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Aug 04 03:22:04.995809 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:04.995810    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bmdrm\""
Aug 04 03:22:05.001727 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:05.001701    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"]
Aug 04 03:22:05.059125 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:05.059092    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79fbr\" (UniqueName: \"kubernetes.io/projected/d8b40e05-241f-40a8-a72c-310f16ecbe23-kube-api-access-79fbr\") pod \"payload-pre-processing-687dbb6cf6-sg588\" (UID: \"d8b40e05-241f-40a8-a72c-310f16ecbe23\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"
Aug 04 03:22:05.059282 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:05.059140    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8b40e05-241f-40a8-a72c-310f16ecbe23-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-sg588\" (UID: \"d8b40e05-241f-40a8-a72c-310f16ecbe23\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"
Aug 04 03:22:05.159801 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:05.159716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8b40e05-241f-40a8-a72c-310f16ecbe23-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-sg588\" (UID: \"d8b40e05-241f-40a8-a72c-310f16ecbe23\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"
Aug 04 03:22:05.160230 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:05.159865    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79fbr\" (UniqueName: \"kubernetes.io/projected/d8b40e05-241f-40a8-a72c-310f16ecbe23-kube-api-access-79fbr\") pod \"payload-pre-processing-687dbb6cf6-sg588\" (UID: \"d8b40e05-241f-40a8-a72c-310f16ecbe23\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"
Aug 04 03:22:05.160425 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:05.160405    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8b40e05-241f-40a8-a72c-310f16ecbe23-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-sg588\" (UID: \"d8b40e05-241f-40a8-a72c-310f16ecbe23\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"
Aug 04 03:22:05.167305 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:05.167277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79fbr\" (UniqueName: \"kubernetes.io/projected/d8b40e05-241f-40a8-a72c-310f16ecbe23-kube-api-access-79fbr\") pod \"payload-pre-processing-687dbb6cf6-sg588\" (UID: \"d8b40e05-241f-40a8-a72c-310f16ecbe23\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"
Aug 04 03:22:05.305149 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:05.305113    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"
Aug 04 03:22:05.441115 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:05.441084    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"]
Aug 04 03:22:05.443361 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:22:05.443335    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b40e05_241f_40a8_a72c_310f16ecbe23.slice/crio-3344b308db8b7072d059ed84c3d980fa6b5e33d1015c5a04e908510da5adcc2a WatchSource:0}: Error finding container 3344b308db8b7072d059ed84c3d980fa6b5e33d1015c5a04e908510da5adcc2a: Status 404 returned error can't find the container with id 3344b308db8b7072d059ed84c3d980fa6b5e33d1015c5a04e908510da5adcc2a
Aug 04 03:22:05.999243 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:05.999208    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588" event={"ID":"d8b40e05-241f-40a8-a72c-310f16ecbe23","Type":"ContainerStarted","Data":"3344b308db8b7072d059ed84c3d980fa6b5e33d1015c5a04e908510da5adcc2a"}
Aug 04 03:22:09.014353 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:09.014313    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588" event={"ID":"d8b40e05-241f-40a8-a72c-310f16ecbe23","Type":"ContainerStarted","Data":"6bd4d040e966aa260231fd0ef8038828a35f7ca126c2466f7df7a8cc47858191"}
Aug 04 03:22:09.014759 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:09.014446    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"
Aug 04 03:22:09.031112 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:09.031046    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588" podStartSLOduration=2.208434415 podStartE2EDuration="5.031026693s" totalImagesPullingTime="2.822592278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:22:09.030403189 +0000 UTC m=+746.328543671" watchObservedRunningTime="2026-08-04 03:22:09.031026693 +0000 UTC m=+746.329167184"
Aug 04 03:22:13.036799 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:13.036764    2559 generic.go:356] "Generic (PLEG): container finished" podID="6f328e85-83a1-461f-913b-d9d77efb5bf9" containerID="72d491f0b07e10ef456d0ac3a1fa3058763b0e63862157fed8255fa8bc61e47a" exitCode=0
Aug 04 03:22:13.037311 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:13.036840    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz" event={"ID":"6f328e85-83a1-461f-913b-d9d77efb5bf9","Type":"ContainerDied","Data":"72d491f0b07e10ef456d0ac3a1fa3058763b0e63862157fed8255fa8bc61e47a"}
Aug 04 03:22:13.037311 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:13.037307    2559 scope.go:117] "RemoveContainer" containerID="72d491f0b07e10ef456d0ac3a1fa3058763b0e63862157fed8255fa8bc61e47a"
Aug 04 03:22:13.038379 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:13.038315    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:22:13.038449 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:13.038396    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:22:13.038483 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:13.038445    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:22:13.733839 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:13.733801    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:14.045626 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:14.045504    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz" event={"ID":"6f328e85-83a1-461f-913b-d9d77efb5bf9","Type":"ContainerStarted","Data":"dfbe9075c2ec99f0d6aff164ed9068d16b293e737cc6203152342cec136e381d"}
Aug 04 03:22:14.215874 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:14.215837    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:22:14.216096 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:14.216073    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cbzlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Aug 04 03:22:14.216172 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:14.216143    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz" podUID="6f328e85-83a1-461f-913b-d9d77efb5bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Aug 04 03:22:14.440322 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:14.440229    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:14.537248 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:14.537215    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:15.215772 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:15.215731    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cbzlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Aug 04 03:22:15.216162 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:15.215788    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz" podUID="6f328e85-83a1-461f-913b-d9d77efb5bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Aug 04 03:22:16.216111 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:16.216075    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cbzlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Aug 04 03:22:16.216499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:16.216129    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz" podUID="6f328e85-83a1-461f-913b-d9d77efb5bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Aug 04 03:22:17.215665 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:17.215620    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-cbzlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Aug 04 03:22:17.215848 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:17.215684    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz" podUID="6f328e85-83a1-461f-913b-d9d77efb5bf9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Aug 04 03:22:18.219774 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:18.219744    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:22:18.220221 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:18.220027    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:22:18.220655 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:18.220638    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cbzlz"
Aug 04 03:22:20.022449 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:20.022402    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588" podUID="d8b40e05-241f-40a8-a72c-310f16ecbe23" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.37:9004: connect: connection refused"
Aug 04 03:22:25.306322 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:25.306279    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588" podUID="d8b40e05-241f-40a8-a72c-310f16ecbe23" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.37:9004: connect: connection refused"
Aug 04 03:22:29.088465 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:29.088429    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:29.160127 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:29.160048    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:30.022284 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:30.022251    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-sg588"
Aug 04 03:22:37.582972 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.582871    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"]
Aug 04 03:22:37.586856 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.586823    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.589326 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.589298    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 04 03:22:37.590110 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.590090    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jmw7d\""
Aug 04 03:22:37.590260 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.590244    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 04 03:22:37.590341 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.590251    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Aug 04 03:22:37.595717 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.595690    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"]
Aug 04 03:22:37.609724 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.609683    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:22:37.653970 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.653927    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.653970 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.653965    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcdjr\" (UniqueName: \"kubernetes.io/projected/ed2d64db-9822-49b4-9c97-052c175ad3df-kube-api-access-zcdjr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.654181 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.653987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.654181 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.654052    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.654181 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.654098    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.654181 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.654146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2d64db-9822-49b4-9c97-052c175ad3df-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.755281 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.755241    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2d64db-9822-49b4-9c97-052c175ad3df-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.755530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.755309    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.755530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.755328    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcdjr\" (UniqueName: \"kubernetes.io/projected/ed2d64db-9822-49b4-9c97-052c175ad3df-kube-api-access-zcdjr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.755530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.755346    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.755530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.755373    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.755530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.755399    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.755825 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.755765    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.755876 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.755833    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.755967 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.755904    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.757626 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.757603    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2d64db-9822-49b4-9c97-052c175ad3df-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.757873 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.757855    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2d64db-9822-49b4-9c97-052c175ad3df-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.763300 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.763274    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcdjr\" (UniqueName: \"kubernetes.io/projected/ed2d64db-9822-49b4-9c97-052c175ad3df-kube-api-access-zcdjr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg\" (UID: \"ed2d64db-9822-49b4-9c97-052c175ad3df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:37.898272 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:37.898174    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:38.043589 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:38.043567    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"]
Aug 04 03:22:38.046390 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:22:38.046359    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded2d64db_9822_49b4_9c97_052c175ad3df.slice/crio-b9f7544e7369bca6b8eb79df91b5f1fa1a182d517a4ddcf6c0f121cb466479f7 WatchSource:0}: Error finding container b9f7544e7369bca6b8eb79df91b5f1fa1a182d517a4ddcf6c0f121cb466479f7: Status 404 returned error can't find the container with id b9f7544e7369bca6b8eb79df91b5f1fa1a182d517a4ddcf6c0f121cb466479f7
Aug 04 03:22:38.161607 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:38.161493    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg" event={"ID":"ed2d64db-9822-49b4-9c97-052c175ad3df","Type":"ContainerStarted","Data":"b9f7544e7369bca6b8eb79df91b5f1fa1a182d517a4ddcf6c0f121cb466479f7"}
Aug 04 03:22:39.167789 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:39.167755    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg" event={"ID":"ed2d64db-9822-49b4-9c97-052c175ad3df","Type":"ContainerStarted","Data":"abde8978c9d83cf9b1b2596bccad89d62b5ab7459db8071bea20a134c4005037"}
Aug 04 03:22:39.185795 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:39.185734    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg" podStartSLOduration=1.142858958 podStartE2EDuration="2.185712787s" totalImagesPullingTime="1.042853829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:22:39.185201878 +0000 UTC m=+776.483342390" watchObservedRunningTime="2026-08-04 03:22:39.185712787 +0000 UTC m=+776.483853302"
Aug 04 03:22:47.899161 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:47.899119    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:47.899799 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:47.899218    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:47.911771 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:47.911742    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:48.222526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:48.222436    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg"
Aug 04 03:22:56.913118 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:22:56.913077    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:23:09.108060 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:23:09.108014    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:23:24.108759 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:23:24.108721    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:23:47.818516 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:23:47.818476    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:24:00.778420 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:24:00.778383    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:24:34.569694 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:24:34.569659    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:25:28.009402 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.009366    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7885dbcc6c-gwc8b"]
Aug 04 03:25:28.012949 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.012926    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7885dbcc6c-gwc8b"
Aug 04 03:25:28.020190 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.020168    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7885dbcc6c-gwc8b"]
Aug 04 03:25:28.098023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.097985    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/64187948-6905-4ed9-a507-2c696c71ef90-tls-cert\") pod \"authorino-7885dbcc6c-gwc8b\" (UID: \"64187948-6905-4ed9-a507-2c696c71ef90\") " pod="kuadrant-system/authorino-7885dbcc6c-gwc8b"
Aug 04 03:25:28.098201 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.098052    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8fph\" (UniqueName: \"kubernetes.io/projected/64187948-6905-4ed9-a507-2c696c71ef90-kube-api-access-n8fph\") pod \"authorino-7885dbcc6c-gwc8b\" (UID: \"64187948-6905-4ed9-a507-2c696c71ef90\") " pod="kuadrant-system/authorino-7885dbcc6c-gwc8b"
Aug 04 03:25:28.198525 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.198475    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8fph\" (UniqueName: \"kubernetes.io/projected/64187948-6905-4ed9-a507-2c696c71ef90-kube-api-access-n8fph\") pod \"authorino-7885dbcc6c-gwc8b\" (UID: \"64187948-6905-4ed9-a507-2c696c71ef90\") " pod="kuadrant-system/authorino-7885dbcc6c-gwc8b"
Aug 04 03:25:28.198744 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.198611    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/64187948-6905-4ed9-a507-2c696c71ef90-tls-cert\") pod \"authorino-7885dbcc6c-gwc8b\" (UID: \"64187948-6905-4ed9-a507-2c696c71ef90\") " pod="kuadrant-system/authorino-7885dbcc6c-gwc8b"
Aug 04 03:25:28.201022 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.200999    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/64187948-6905-4ed9-a507-2c696c71ef90-tls-cert\") pod \"authorino-7885dbcc6c-gwc8b\" (UID: \"64187948-6905-4ed9-a507-2c696c71ef90\") " pod="kuadrant-system/authorino-7885dbcc6c-gwc8b"
Aug 04 03:25:28.205712 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.205691    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8fph\" (UniqueName: \"kubernetes.io/projected/64187948-6905-4ed9-a507-2c696c71ef90-kube-api-access-n8fph\") pod \"authorino-7885dbcc6c-gwc8b\" (UID: \"64187948-6905-4ed9-a507-2c696c71ef90\") " pod="kuadrant-system/authorino-7885dbcc6c-gwc8b"
Aug 04 03:25:28.322754 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.322669    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7885dbcc6c-gwc8b"
Aug 04 03:25:28.392204 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.392135    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:25:28.457705 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.457638    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7885dbcc6c-gwc8b"]
Aug 04 03:25:28.459904 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.459847    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:25:28.460348 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:25:28.460317    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64187948_6905_4ed9_a507_2c696c71ef90.slice/crio-b72fcc44f5a7f04773c39d01672723d52eebb132ed54a53603587d2c2d3c4330 WatchSource:0}: Error finding container b72fcc44f5a7f04773c39d01672723d52eebb132ed54a53603587d2c2d3c4330: Status 404 returned error can't find the container with id b72fcc44f5a7f04773c39d01672723d52eebb132ed54a53603587d2c2d3c4330
Aug 04 03:25:28.974739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:28.974704    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7885dbcc6c-gwc8b" event={"ID":"64187948-6905-4ed9-a507-2c696c71ef90","Type":"ContainerStarted","Data":"b72fcc44f5a7f04773c39d01672723d52eebb132ed54a53603587d2c2d3c4330"}
Aug 04 03:25:29.506096 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:29.506052    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:25:29.639013 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:29.638972    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:25:29.982559 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:29.982487    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7885dbcc6c-gwc8b" event={"ID":"64187948-6905-4ed9-a507-2c696c71ef90","Type":"ContainerStarted","Data":"895929e3d942aa953bee50ad91afc9d17e4f78a2f3d77c7250cd5cc953d46c99"}
Aug 04 03:25:29.998844 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:29.998778    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7885dbcc6c-gwc8b" podStartSLOduration=2.506483998 podStartE2EDuration="2.998754432s" totalImagesPullingTime="492.270434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:25:29.997804601 +0000 UTC m=+947.295945083" watchObservedRunningTime="2026-08-04 03:25:29.998754432 +0000 UTC m=+947.296894921"
Aug 04 03:25:30.042249 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.042216    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b55cb4b98-lvtt6"]
Aug 04 03:25:30.042469 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.042441    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7b55cb4b98-lvtt6" podUID="8ce70c95-6011-461e-af2e-61b8c202065a" containerName="authorino" containerID="cri-o://1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a" gracePeriod=30
Aug 04 03:25:30.310932 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.310904    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b55cb4b98-lvtt6"
Aug 04 03:25:30.319381 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.319354    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ce70c95-6011-461e-af2e-61b8c202065a-tls-cert\") pod \"8ce70c95-6011-461e-af2e-61b8c202065a\" (UID: \"8ce70c95-6011-461e-af2e-61b8c202065a\") "
Aug 04 03:25:30.319535 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.319427    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58w9j\" (UniqueName: \"kubernetes.io/projected/8ce70c95-6011-461e-af2e-61b8c202065a-kube-api-access-58w9j\") pod \"8ce70c95-6011-461e-af2e-61b8c202065a\" (UID: \"8ce70c95-6011-461e-af2e-61b8c202065a\") "
Aug 04 03:25:30.321682 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.321649    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ce70c95-6011-461e-af2e-61b8c202065a-kube-api-access-58w9j" (OuterVolumeSpecName: "kube-api-access-58w9j") pod "8ce70c95-6011-461e-af2e-61b8c202065a" (UID: "8ce70c95-6011-461e-af2e-61b8c202065a"). InnerVolumeSpecName "kube-api-access-58w9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:25:30.337651 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.337615    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ce70c95-6011-461e-af2e-61b8c202065a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "8ce70c95-6011-461e-af2e-61b8c202065a" (UID: "8ce70c95-6011-461e-af2e-61b8c202065a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 03:25:30.421219 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.421184    2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ce70c95-6011-461e-af2e-61b8c202065a-tls-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:25:30.421219 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.421215    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58w9j\" (UniqueName: \"kubernetes.io/projected/8ce70c95-6011-461e-af2e-61b8c202065a-kube-api-access-58w9j\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:25:30.989493 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.989462    2559 generic.go:356] "Generic (PLEG): container finished" podID="8ce70c95-6011-461e-af2e-61b8c202065a" containerID="1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a" exitCode=0
Aug 04 03:25:30.989987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.989509    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b55cb4b98-lvtt6"
Aug 04 03:25:30.989987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.989580    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b55cb4b98-lvtt6" event={"ID":"8ce70c95-6011-461e-af2e-61b8c202065a","Type":"ContainerDied","Data":"1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a"}
Aug 04 03:25:30.989987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.989628    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b55cb4b98-lvtt6" event={"ID":"8ce70c95-6011-461e-af2e-61b8c202065a","Type":"ContainerDied","Data":"cb01146500f9d50fedb8649cf2f92934eed6524cccb2ecb108a344e4b5da2833"}
Aug 04 03:25:30.989987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.989653    2559 scope.go:117] "RemoveContainer" containerID="1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a"
Aug 04 03:25:30.998910 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.998881    2559 scope.go:117] "RemoveContainer" containerID="1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a"
Aug 04 03:25:30.999227 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:25:30.999191    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a\": container with ID starting with 1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a not found: ID does not exist" containerID="1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a"
Aug 04 03:25:30.999324 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:30.999220    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a"} err="failed to get container status \"1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a\": rpc error: code = NotFound desc = could not find container \"1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a\": container with ID starting with 1c247e26f15cae4454c4ac3b6612726b38fe7c2cadc9f03d6044b6c78c14fe5a not found: ID does not exist"
Aug 04 03:25:31.012353 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:31.012313    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b55cb4b98-lvtt6"]
Aug 04 03:25:31.022444 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:31.022412    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b55cb4b98-lvtt6"]
Aug 04 03:25:31.239237 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:25:31.239202    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ce70c95-6011-461e-af2e-61b8c202065a" path="/var/lib/kubelet/pods/8ce70c95-6011-461e-af2e-61b8c202065a/volumes"
Aug 04 03:27:32.944193 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:27:32.944154    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:27:34.125511 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:27:34.125478    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:27:59.195833 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:27:59.195736    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:27:59.296867 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:27:59.296835    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:28:19.688847 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:28:19.688814    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:28:53.602367 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:28:53.602327    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:28:53.701940 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:28:53.701912    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:28:53.890269 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:28:53.890188    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:28:59.449018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:28:59.448983    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:28:59.505062 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:28:59.505033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:28:59.600200 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:28:59.600171    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:28:59.700058 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:28:59.699973    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:28:59.800369 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:28:59.800338    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:29:05.690021 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:29:05.689933    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:29:05.790535 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:29:05.790502    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:30:00.147331 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:00.147293    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs"]
Aug 04 03:30:00.147948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:00.147810    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ce70c95-6011-461e-af2e-61b8c202065a" containerName="authorino"
Aug 04 03:30:00.147948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:00.147830    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ce70c95-6011-461e-af2e-61b8c202065a" containerName="authorino"
Aug 04 03:30:00.147948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:00.147940    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ce70c95-6011-461e-af2e-61b8c202065a" containerName="authorino"
Aug 04 03:30:00.149790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:00.149769    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs"
Aug 04 03:30:00.152069 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:00.152048    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7lrf4\""
Aug 04 03:30:00.158623 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:00.158604    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs"
Aug 04 03:30:00.164616 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:00.164594    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs"]
Aug 04 03:30:00.290151 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:00.290124    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs"]
Aug 04 03:30:00.292932 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:30:00.292907    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69933532_cdc4_43a2_a946_17121b7c7053.slice/crio-a68c6addcff080c3b5b1f32a68b86b5c30d32a0c5f92ef737cfe668735b4811c WatchSource:0}: Error finding container a68c6addcff080c3b5b1f32a68b86b5c30d32a0c5f92ef737cfe668735b4811c: Status 404 returned error can't find the container with id a68c6addcff080c3b5b1f32a68b86b5c30d32a0c5f92ef737cfe668735b4811c
Aug 04 03:30:00.295166 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:00.295149    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 03:30:01.244113 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:01.244081    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs" event={"ID":"69933532-cdc4-43a2-a946-17121b7c7053","Type":"ContainerStarted","Data":"a68c6addcff080c3b5b1f32a68b86b5c30d32a0c5f92ef737cfe668735b4811c"}
Aug 04 03:30:03.254735 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:03.254700    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs" event={"ID":"69933532-cdc4-43a2-a946-17121b7c7053","Type":"ContainerStarted","Data":"c4f9d1f0c56f9194a44f83c2933e07ddcff08e7f9921e89ca0a6f55074a6c9ac"}
Aug 04 03:30:03.269182 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:03.269136    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs" podStartSLOduration=0.874157315 podStartE2EDuration="3.269122452s" totalImagesPullingTime="2.394965137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:30:03.268349188 +0000 UTC m=+1220.566489679" watchObservedRunningTime="2026-08-04 03:30:03.269122452 +0000 UTC m=+1220.567262939"
Aug 04 03:30:04.261211 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:04.261173    2559 generic.go:356] "Generic (PLEG): container finished" podID="69933532-cdc4-43a2-a946-17121b7c7053" containerID="c4f9d1f0c56f9194a44f83c2933e07ddcff08e7f9921e89ca0a6f55074a6c9ac" exitCode=0
Aug 04 03:30:04.261601 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:04.261235    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs" event={"ID":"69933532-cdc4-43a2-a946-17121b7c7053","Type":"ContainerDied","Data":"c4f9d1f0c56f9194a44f83c2933e07ddcff08e7f9921e89ca0a6f55074a6c9ac"}
Aug 04 03:30:05.393905 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:05.393881    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs"
Aug 04 03:30:06.271108 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:06.271073    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs" event={"ID":"69933532-cdc4-43a2-a946-17121b7c7053","Type":"ContainerDied","Data":"a68c6addcff080c3b5b1f32a68b86b5c30d32a0c5f92ef737cfe668735b4811c"}
Aug 04 03:30:06.271108 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:06.271093    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs"
Aug 04 03:30:06.271348 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:06.271124    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a68c6addcff080c3b5b1f32a68b86b5c30d32a0c5f92ef737cfe668735b4811c"
Aug 04 03:30:43.615321 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:30:43.615229    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:32:33.488711 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:32:33.488675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:35:05.300690 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.300603    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"]
Aug 04 03:35:05.301254 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.300828    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4" podUID="d658da66-34f8-45e6-808a-a4b946939e09" containerName="manager" containerID="cri-o://ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30" gracePeriod=10
Aug 04 03:35:05.549058 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.549032    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:35:05.629039 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.628959    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d658da66-34f8-45e6-808a-a4b946939e09-extensions-socket-volume\") pod \"d658da66-34f8-45e6-808a-a4b946939e09\" (UID: \"d658da66-34f8-45e6-808a-a4b946939e09\") "
Aug 04 03:35:05.629039 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.629012    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2k7z\" (UniqueName: \"kubernetes.io/projected/d658da66-34f8-45e6-808a-a4b946939e09-kube-api-access-w2k7z\") pod \"d658da66-34f8-45e6-808a-a4b946939e09\" (UID: \"d658da66-34f8-45e6-808a-a4b946939e09\") "
Aug 04 03:35:05.629387 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.629367    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d658da66-34f8-45e6-808a-a4b946939e09-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d658da66-34f8-45e6-808a-a4b946939e09" (UID: "d658da66-34f8-45e6-808a-a4b946939e09"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:35:05.630929 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.630909    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d658da66-34f8-45e6-808a-a4b946939e09-kube-api-access-w2k7z" (OuterVolumeSpecName: "kube-api-access-w2k7z") pod "d658da66-34f8-45e6-808a-a4b946939e09" (UID: "d658da66-34f8-45e6-808a-a4b946939e09"). InnerVolumeSpecName "kube-api-access-w2k7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:35:05.662202 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.662172    2559 generic.go:356] "Generic (PLEG): container finished" podID="d658da66-34f8-45e6-808a-a4b946939e09" containerID="ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30" exitCode=0
Aug 04 03:35:05.662329 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.662228    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"
Aug 04 03:35:05.662329 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.662255    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4" event={"ID":"d658da66-34f8-45e6-808a-a4b946939e09","Type":"ContainerDied","Data":"ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30"}
Aug 04 03:35:05.662402 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.662329    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4" event={"ID":"d658da66-34f8-45e6-808a-a4b946939e09","Type":"ContainerDied","Data":"78591c31c175af88886b8c4b64814b8c8aa191374bdb80787166e6cc6bbddc9a"}
Aug 04 03:35:05.662402 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.662351    2559 scope.go:117] "RemoveContainer" containerID="ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30"
Aug 04 03:35:05.672472 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.672456    2559 scope.go:117] "RemoveContainer" containerID="ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30"
Aug 04 03:35:05.672798 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:35:05.672774    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30\": container with ID starting with ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30 not found: ID does not exist" containerID="ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30"
Aug 04 03:35:05.672844 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.672811    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30"} err="failed to get container status \"ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30\": rpc error: code = NotFound desc = could not find container \"ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30\": container with ID starting with ab36e3c7afe356ca7b3118be7e47da7f03255698802118cc4f8cc76ecd3b4a30 not found: ID does not exist"
Aug 04 03:35:05.683057 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.683032    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"]
Aug 04 03:35:05.685904 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.685880    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4n4r4"]
Aug 04 03:35:05.729905 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.729875    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d658da66-34f8-45e6-808a-a4b946939e09-extensions-socket-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:35:05.729905 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:05.729901    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2k7z\" (UniqueName: \"kubernetes.io/projected/d658da66-34f8-45e6-808a-a4b946939e09-kube-api-access-w2k7z\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:35:07.238519 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:35:07.238485    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d658da66-34f8-45e6-808a-a4b946939e09" path="/var/lib/kubelet/pods/d658da66-34f8-45e6-808a-a4b946939e09/volumes"
Aug 04 03:36:05.970741 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:05.970706    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"]
Aug 04 03:36:05.971116 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:05.971056    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69933532-cdc4-43a2-a946-17121b7c7053" containerName="cleanup"
Aug 04 03:36:05.971116 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:05.971072    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="69933532-cdc4-43a2-a946-17121b7c7053" containerName="cleanup"
Aug 04 03:36:05.971116 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:05.971088    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d658da66-34f8-45e6-808a-a4b946939e09" containerName="manager"
Aug 04 03:36:05.971116 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:05.971093    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d658da66-34f8-45e6-808a-a4b946939e09" containerName="manager"
Aug 04 03:36:05.971236 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:05.971147    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d658da66-34f8-45e6-808a-a4b946939e09" containerName="manager"
Aug 04 03:36:05.971236 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:05.971158    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="69933532-cdc4-43a2-a946-17121b7c7053" containerName="cleanup"
Aug 04 03:36:05.975290 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:05.975274    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:36:05.977909 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:05.977885    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rgf8h\""
Aug 04 03:36:05.983493 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:05.983469    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"]
Aug 04 03:36:06.079026 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.078998    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e559706b-fe79-46cf-9998-b0a331522a8c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4qn6v\" (UID: \"e559706b-fe79-46cf-9998-b0a331522a8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:36:06.079026 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.079031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wvn5\" (UniqueName: \"kubernetes.io/projected/e559706b-fe79-46cf-9998-b0a331522a8c-kube-api-access-8wvn5\") pod \"kuadrant-operator-controller-manager-86878c769f-4qn6v\" (UID: \"e559706b-fe79-46cf-9998-b0a331522a8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:36:06.180381 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.180351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e559706b-fe79-46cf-9998-b0a331522a8c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4qn6v\" (UID: \"e559706b-fe79-46cf-9998-b0a331522a8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:36:06.180381 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.180387    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wvn5\" (UniqueName: \"kubernetes.io/projected/e559706b-fe79-46cf-9998-b0a331522a8c-kube-api-access-8wvn5\") pod \"kuadrant-operator-controller-manager-86878c769f-4qn6v\" (UID: \"e559706b-fe79-46cf-9998-b0a331522a8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:36:06.180771 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.180753    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e559706b-fe79-46cf-9998-b0a331522a8c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4qn6v\" (UID: \"e559706b-fe79-46cf-9998-b0a331522a8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:36:06.188453 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.188430    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wvn5\" (UniqueName: \"kubernetes.io/projected/e559706b-fe79-46cf-9998-b0a331522a8c-kube-api-access-8wvn5\") pod \"kuadrant-operator-controller-manager-86878c769f-4qn6v\" (UID: \"e559706b-fe79-46cf-9998-b0a331522a8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:36:06.286257 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.286231    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:36:06.414978 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.414810    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"]
Aug 04 03:36:06.417317 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:36:06.417288    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode559706b_fe79_46cf_9998_b0a331522a8c.slice/crio-0694b4b72cecb4d712083e5fccb6b5cb0ab72e8f58d67f984687be93699abe2b WatchSource:0}: Error finding container 0694b4b72cecb4d712083e5fccb6b5cb0ab72e8f58d67f984687be93699abe2b: Status 404 returned error can't find the container with id 0694b4b72cecb4d712083e5fccb6b5cb0ab72e8f58d67f984687be93699abe2b
Aug 04 03:36:06.939100 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.939069    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v" event={"ID":"e559706b-fe79-46cf-9998-b0a331522a8c","Type":"ContainerStarted","Data":"aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5"}
Aug 04 03:36:06.939258 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.939145    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v" event={"ID":"e559706b-fe79-46cf-9998-b0a331522a8c","Type":"ContainerStarted","Data":"0694b4b72cecb4d712083e5fccb6b5cb0ab72e8f58d67f984687be93699abe2b"}
Aug 04 03:36:06.939258 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.939205    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:36:06.958377 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:06.958308    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v" podStartSLOduration=1.9582900890000001 podStartE2EDuration="1.958290089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:36:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:36:06.956757788 +0000 UTC m=+1584.254898321" watchObservedRunningTime="2026-08-04 03:36:06.958290089 +0000 UTC m=+1584.256430576"
Aug 04 03:36:17.945956 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:36:17.945922    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:37:58.704556 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:37:58.704507    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:39:20.097235 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.097206    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"]
Aug 04 03:39:20.100408 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.100387    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.102780 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.102762    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-44fc10de-kserve-self-signed-certs\""
Aug 04 03:39:20.119536 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.119516    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"]
Aug 04 03:39:20.131986 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.131956    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:39:20.196964 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.196931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-home\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.197113 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.196987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/932058f3-40b8-45ad-a778-7d61d80a9564-tls-certs\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.197113 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.197053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-tmp-dir\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.197113 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.197085    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt79z\" (UniqueName: \"kubernetes.io/projected/932058f3-40b8-45ad-a778-7d61d80a9564-kube-api-access-qt79z\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.197278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.197117    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-model-cache\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.197278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.197145    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-dshm\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.298685 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.298648    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-home\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.298877 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.298777    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/932058f3-40b8-45ad-a778-7d61d80a9564-tls-certs\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.298877 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.298821    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-tmp-dir\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.299044 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.298937    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt79z\" (UniqueName: \"kubernetes.io/projected/932058f3-40b8-45ad-a778-7d61d80a9564-kube-api-access-qt79z\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.299044 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.298974    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-home\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.299044 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.298991    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-model-cache\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.299044 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.299038    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-dshm\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.299234 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.299126    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-tmp-dir\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.299234 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.299189    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-model-cache\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.301103 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.301072    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-dshm\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.301268 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.301249    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/932058f3-40b8-45ad-a778-7d61d80a9564-tls-certs\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.306389 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.306369    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt79z\" (UniqueName: \"kubernetes.io/projected/932058f3-40b8-45ad-a778-7d61d80a9564-kube-api-access-qt79z\") pod \"e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") " pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.409104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.409030    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:20.546676 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.546649    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"]
Aug 04 03:39:20.548883 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:39:20.548855    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod932058f3_40b8_45ad_a778_7d61d80a9564.slice/crio-d0995b72f388b8299a47e8fd3007f8459e5904d26d0b1909cfa2bcc22892b090 WatchSource:0}: Error finding container d0995b72f388b8299a47e8fd3007f8459e5904d26d0b1909cfa2bcc22892b090: Status 404 returned error can't find the container with id d0995b72f388b8299a47e8fd3007f8459e5904d26d0b1909cfa2bcc22892b090
Aug 04 03:39:20.829310 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.829270    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz" event={"ID":"932058f3-40b8-45ad-a778-7d61d80a9564","Type":"ContainerStarted","Data":"feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572"}
Aug 04 03:39:20.829310 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.829312    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz" event={"ID":"932058f3-40b8-45ad-a778-7d61d80a9564","Type":"ContainerStarted","Data":"d0995b72f388b8299a47e8fd3007f8459e5904d26d0b1909cfa2bcc22892b090"}
Aug 04 03:39:20.845922 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:20.845867    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz" podStartSLOduration=0.845848864 podStartE2EDuration="845.848864ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:39:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:39:20.845279069 +0000 UTC m=+1778.143419563" watchObservedRunningTime="2026-08-04 03:39:20.845848864 +0000 UTC m=+1778.143989352"
Aug 04 03:39:30.409344 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:30.409271    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:30.409741 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:30.409348    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:30.421980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:30.421957    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:30.893633 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:30.893606    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:39:33.689008 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.688974    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"]
Aug 04 03:39:33.692718 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.692693    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.696167 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.696147    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-44fc10de-kserve-self-signed-certs\""
Aug 04 03:39:33.702799 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.702779    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"]
Aug 04 03:39:33.713924 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.713897    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h4rm\" (UniqueName: \"kubernetes.io/projected/31181f88-d599-4c3b-a413-aff98db23f90-kube-api-access-6h4rm\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.714034 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.713989    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:39:33.714034 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.714020    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-home\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.714143 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.714043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-model-cache\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.714143 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.714098    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-tmp-dir\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.714254 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.714147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31181f88-d599-4c3b-a413-aff98db23f90-tls-certs\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.714254 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.714193    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-dshm\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.815378 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.815349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-dshm\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.815604 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.815396    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h4rm\" (UniqueName: \"kubernetes.io/projected/31181f88-d599-4c3b-a413-aff98db23f90-kube-api-access-6h4rm\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.815604 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.815435    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-home\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.815604 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.815450    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-model-cache\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.815604 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.815483    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-tmp-dir\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.815604 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.815513    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31181f88-d599-4c3b-a413-aff98db23f90-tls-certs\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.815881 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.815843    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-model-cache\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.815919 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.815892    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-home\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.815998 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.815976    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-tmp-dir\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.817741 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.817715    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-dshm\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.817919 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.817905    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31181f88-d599-4c3b-a413-aff98db23f90-tls-certs\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:33.823209 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:33.823189    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h4rm\" (UniqueName: \"kubernetes.io/projected/31181f88-d599-4c3b-a413-aff98db23f90-kube-api-access-6h4rm\") pod \"e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") " pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:34.002183 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:34.002149    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:34.137140 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:34.137069    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"]
Aug 04 03:39:34.138441 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:39:34.138414    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31181f88_d599_4c3b_a413_aff98db23f90.slice/crio-1f4108462d33a2bb27d3c7ef0aa13f0e256bd017d50dcb25536d5d5a09301b62 WatchSource:0}: Error finding container 1f4108462d33a2bb27d3c7ef0aa13f0e256bd017d50dcb25536d5d5a09301b62: Status 404 returned error can't find the container with id 1f4108462d33a2bb27d3c7ef0aa13f0e256bd017d50dcb25536d5d5a09301b62
Aug 04 03:39:34.901223 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:34.901140    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749" event={"ID":"31181f88-d599-4c3b-a413-aff98db23f90","Type":"ContainerStarted","Data":"3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284"}
Aug 04 03:39:34.901223 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:34.901183    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749" event={"ID":"31181f88-d599-4c3b-a413-aff98db23f90","Type":"ContainerStarted","Data":"1f4108462d33a2bb27d3c7ef0aa13f0e256bd017d50dcb25536d5d5a09301b62"}
Aug 04 03:39:34.920581 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:34.920510    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749" podStartSLOduration=1.920495536 podStartE2EDuration="1.920495536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:39:34.919078204 +0000 UTC m=+1792.217218692" watchObservedRunningTime="2026-08-04 03:39:34.920495536 +0000 UTC m=+1792.218636187"
Aug 04 03:39:44.002384 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:44.002346    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:44.002877 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:44.002632    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:44.020005 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:44.019980    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:44.963340 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:44.963308    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:39:56.069850 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:56.069815    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"]
Aug 04 03:39:56.091499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:56.091470    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:39:57.320831 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:57.320799    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"]
Aug 04 03:39:57.342744 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:39:57.342715    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:40:11.088630 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.088590    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz" podUID="932058f3-40b8-45ad-a778-7d61d80a9564" containerName="main" containerID="cri-o://feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572" gracePeriod=45
Aug 04 03:40:11.335576 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.335526    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:40:11.359053 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.358998    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-model-cache\") pod \"932058f3-40b8-45ad-a778-7d61d80a9564\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") "
Aug 04 03:40:11.359053 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.359046    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-home\") pod \"932058f3-40b8-45ad-a778-7d61d80a9564\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") "
Aug 04 03:40:11.359214 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.359086    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-tmp-dir\") pod \"932058f3-40b8-45ad-a778-7d61d80a9564\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") "
Aug 04 03:40:11.359214 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.359127    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qt79z\" (UniqueName: \"kubernetes.io/projected/932058f3-40b8-45ad-a778-7d61d80a9564-kube-api-access-qt79z\") pod \"932058f3-40b8-45ad-a778-7d61d80a9564\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") "
Aug 04 03:40:11.359214 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.359155    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-dshm\") pod \"932058f3-40b8-45ad-a778-7d61d80a9564\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") "
Aug 04 03:40:11.359214 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.359204    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/932058f3-40b8-45ad-a778-7d61d80a9564-tls-certs\") pod \"932058f3-40b8-45ad-a778-7d61d80a9564\" (UID: \"932058f3-40b8-45ad-a778-7d61d80a9564\") "
Aug 04 03:40:11.359425 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.359264    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-model-cache" (OuterVolumeSpecName: "model-cache") pod "932058f3-40b8-45ad-a778-7d61d80a9564" (UID: "932058f3-40b8-45ad-a778-7d61d80a9564"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:40:11.359425 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.359360    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-home" (OuterVolumeSpecName: "home") pod "932058f3-40b8-45ad-a778-7d61d80a9564" (UID: "932058f3-40b8-45ad-a778-7d61d80a9564"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:40:11.359425 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.359368    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "932058f3-40b8-45ad-a778-7d61d80a9564" (UID: "932058f3-40b8-45ad-a778-7d61d80a9564"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:40:11.359620 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.359480    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-model-cache\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:11.359620 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.359493    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-home\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:11.359620 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.359507    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-tmp-dir\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:11.361598 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.361439    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-dshm" (OuterVolumeSpecName: "dshm") pod "932058f3-40b8-45ad-a778-7d61d80a9564" (UID: "932058f3-40b8-45ad-a778-7d61d80a9564"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:40:11.361598 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.361522    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/932058f3-40b8-45ad-a778-7d61d80a9564-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "932058f3-40b8-45ad-a778-7d61d80a9564" (UID: "932058f3-40b8-45ad-a778-7d61d80a9564"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 03:40:11.361766 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.361621    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/932058f3-40b8-45ad-a778-7d61d80a9564-kube-api-access-qt79z" (OuterVolumeSpecName: "kube-api-access-qt79z") pod "932058f3-40b8-45ad-a778-7d61d80a9564" (UID: "932058f3-40b8-45ad-a778-7d61d80a9564"). InnerVolumeSpecName "kube-api-access-qt79z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:40:11.460530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.460497    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt79z\" (UniqueName: \"kubernetes.io/projected/932058f3-40b8-45ad-a778-7d61d80a9564-kube-api-access-qt79z\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:11.460530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.460522    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/932058f3-40b8-45ad-a778-7d61d80a9564-dshm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:11.460530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:11.460533    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/932058f3-40b8-45ad-a778-7d61d80a9564-tls-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:12.077271 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.077231    2559 generic.go:356] "Generic (PLEG): container finished" podID="932058f3-40b8-45ad-a778-7d61d80a9564" containerID="feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572" exitCode=0
Aug 04 03:40:12.077437 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.077279    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"
Aug 04 03:40:12.077437 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.077297    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz" event={"ID":"932058f3-40b8-45ad-a778-7d61d80a9564","Type":"ContainerDied","Data":"feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572"}
Aug 04 03:40:12.077437 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.077337    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz" event={"ID":"932058f3-40b8-45ad-a778-7d61d80a9564","Type":"ContainerDied","Data":"d0995b72f388b8299a47e8fd3007f8459e5904d26d0b1909cfa2bcc22892b090"}
Aug 04 03:40:12.077437 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.077359    2559 scope.go:117] "RemoveContainer" containerID="feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572"
Aug 04 03:40:12.090062 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.089852    2559 scope.go:117] "RemoveContainer" containerID="feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572"
Aug 04 03:40:12.090351 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:40:12.090257    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572\": container with ID starting with feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572 not found: ID does not exist" containerID="feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572"
Aug 04 03:40:12.090351 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.090290    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572"} err="failed to get container status \"feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572\": rpc error: code = NotFound desc = could not find container \"feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572\": container with ID starting with feccc02d7c6939cbc9a0b85dadfafd96cb7e49abca2f3da6d0c6fdb051762572 not found: ID does not exist"
Aug 04 03:40:12.118835 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.118807    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"]
Aug 04 03:40:12.143431 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.143404    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-44fc10de-kserve-769b98ffc9-xk5nz"]
Aug 04 03:40:12.340336 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.340256    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749" podUID="31181f88-d599-4c3b-a413-aff98db23f90" containerName="main" containerID="cri-o://3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284" gracePeriod=45
Aug 04 03:40:12.588784 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.588763    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:40:12.669804 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.669731    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-model-cache\") pod \"31181f88-d599-4c3b-a413-aff98db23f90\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") "
Aug 04 03:40:12.669804 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.669789    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-dshm\") pod \"31181f88-d599-4c3b-a413-aff98db23f90\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") "
Aug 04 03:40:12.670002 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.669820    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6h4rm\" (UniqueName: \"kubernetes.io/projected/31181f88-d599-4c3b-a413-aff98db23f90-kube-api-access-6h4rm\") pod \"31181f88-d599-4c3b-a413-aff98db23f90\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") "
Aug 04 03:40:12.670002 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.669847    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-home\") pod \"31181f88-d599-4c3b-a413-aff98db23f90\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") "
Aug 04 03:40:12.670002 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.669862    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-tmp-dir\") pod \"31181f88-d599-4c3b-a413-aff98db23f90\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") "
Aug 04 03:40:12.670002 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.669972    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31181f88-d599-4c3b-a413-aff98db23f90-tls-certs\") pod \"31181f88-d599-4c3b-a413-aff98db23f90\" (UID: \"31181f88-d599-4c3b-a413-aff98db23f90\") "
Aug 04 03:40:12.670002 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.669996    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-model-cache" (OuterVolumeSpecName: "model-cache") pod "31181f88-d599-4c3b-a413-aff98db23f90" (UID: "31181f88-d599-4c3b-a413-aff98db23f90"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:40:12.670243 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.670095    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-home" (OuterVolumeSpecName: "home") pod "31181f88-d599-4c3b-a413-aff98db23f90" (UID: "31181f88-d599-4c3b-a413-aff98db23f90"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:40:12.670243 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.670129    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "31181f88-d599-4c3b-a413-aff98db23f90" (UID: "31181f88-d599-4c3b-a413-aff98db23f90"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:40:12.670319 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.670277    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-home\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:12.670319 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.670292    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-tmp-dir\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:12.670319 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.670312    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-model-cache\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:12.671905 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.671886    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-dshm" (OuterVolumeSpecName: "dshm") pod "31181f88-d599-4c3b-a413-aff98db23f90" (UID: "31181f88-d599-4c3b-a413-aff98db23f90"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:40:12.671983 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.671934    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31181f88-d599-4c3b-a413-aff98db23f90-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "31181f88-d599-4c3b-a413-aff98db23f90" (UID: "31181f88-d599-4c3b-a413-aff98db23f90"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 03:40:12.672030 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.671989    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31181f88-d599-4c3b-a413-aff98db23f90-kube-api-access-6h4rm" (OuterVolumeSpecName: "kube-api-access-6h4rm") pod "31181f88-d599-4c3b-a413-aff98db23f90" (UID: "31181f88-d599-4c3b-a413-aff98db23f90"). InnerVolumeSpecName "kube-api-access-6h4rm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:40:12.771652 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.771620    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6h4rm\" (UniqueName: \"kubernetes.io/projected/31181f88-d599-4c3b-a413-aff98db23f90-kube-api-access-6h4rm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:12.771652 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.771649    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31181f88-d599-4c3b-a413-aff98db23f90-tls-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:12.771652 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:12.771659    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31181f88-d599-4c3b-a413-aff98db23f90-dshm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:40:13.084356 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:13.084320    2559 generic.go:356] "Generic (PLEG): container finished" podID="31181f88-d599-4c3b-a413-aff98db23f90" containerID="3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284" exitCode=0
Aug 04 03:40:13.084356 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:13.084360    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749" event={"ID":"31181f88-d599-4c3b-a413-aff98db23f90","Type":"ContainerDied","Data":"3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284"}
Aug 04 03:40:13.084613 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:13.084373    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"
Aug 04 03:40:13.084613 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:13.084386    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749" event={"ID":"31181f88-d599-4c3b-a413-aff98db23f90","Type":"ContainerDied","Data":"1f4108462d33a2bb27d3c7ef0aa13f0e256bd017d50dcb25536d5d5a09301b62"}
Aug 04 03:40:13.084613 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:13.084402    2559 scope.go:117] "RemoveContainer" containerID="3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284"
Aug 04 03:40:13.094073 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:13.093906    2559 scope.go:117] "RemoveContainer" containerID="3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284"
Aug 04 03:40:13.094295 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:40:13.094149    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284\": container with ID starting with 3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284 not found: ID does not exist" containerID="3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284"
Aug 04 03:40:13.094295 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:13.094175    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284"} err="failed to get container status \"3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284\": rpc error: code = NotFound desc = could not find container \"3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284\": container with ID starting with 3c8939061806e3353d99547a2932a19c798192d85367d381c99d61851cd25284 not found: ID does not exist"
Aug 04 03:40:13.104969 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:13.104947    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"]
Aug 04 03:40:13.107736 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:13.107715    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-44fc10de-kserve-6d4447dd47-xq749"]
Aug 04 03:40:13.238010 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:13.237975    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31181f88-d599-4c3b-a413-aff98db23f90" path="/var/lib/kubelet/pods/31181f88-d599-4c3b-a413-aff98db23f90/volumes"
Aug 04 03:40:13.238362 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:13.238349    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="932058f3-40b8-45ad-a778-7d61d80a9564" path="/var/lib/kubelet/pods/932058f3-40b8-45ad-a778-7d61d80a9564/volumes"
Aug 04 03:40:57.351313 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:40:57.351270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:42:31.700143 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:42:31.700110    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:43:59.540199 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:43:59.540146    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:03.639323 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:03.639270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:03.742197 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:03.742158    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:07.960070 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:07.959966    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:35.760142 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:35.760095    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:38.662366 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:38.662325    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:38.762555 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:38.762500    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:39.094636 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:39.094603    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:44.796995 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:44.796958    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:44.897197 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:44.897162    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:45.028691 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.028650    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:45.282904 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.282874    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"]
Aug 04 03:44:45.283281 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.283261    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="932058f3-40b8-45ad-a778-7d61d80a9564" containerName="main"
Aug 04 03:44:45.283341 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.283283    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="932058f3-40b8-45ad-a778-7d61d80a9564" containerName="main"
Aug 04 03:44:45.283341 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.283295    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31181f88-d599-4c3b-a413-aff98db23f90" containerName="main"
Aug 04 03:44:45.283341 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.283301    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="31181f88-d599-4c3b-a413-aff98db23f90" containerName="main"
Aug 04 03:44:45.283447 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.283356    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="31181f88-d599-4c3b-a413-aff98db23f90" containerName="main"
Aug 04 03:44:45.283447 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.283364    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="932058f3-40b8-45ad-a778-7d61d80a9564" containerName="main"
Aug 04 03:44:45.286406 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.286391    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.289217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.289196    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-49bbb41b-openshift-default-dockercfg-cszjh\""
Aug 04 03:44:45.307213 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.307190    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"]
Aug 04 03:44:45.308590 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.308567    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-credential-socket\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.308710 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.308594    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-podinfo\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.308710 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.308622    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r27m7\" (UniqueName: \"kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-kube-api-access-r27m7\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.308710 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.308648    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-certs\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.308895 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.308730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-socket\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.308895 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.308808    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-token\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.308895 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.308851    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istiod-ca-cert\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.308895 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.308884    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-envoy\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.309107 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.308995    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-data\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.322124 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.322092    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:45.409670 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.409637    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-envoy\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.409855 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.409697    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-data\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.409855 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.409726    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-credential-socket\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.409855 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.409743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-podinfo\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.409855 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.409776    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r27m7\" (UniqueName: \"kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-kube-api-access-r27m7\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.409855 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.409800    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-certs\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.409855 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.409818    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-socket\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.409855 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.409854    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-token\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.410242 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.409893    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istiod-ca-cert\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.410242 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.410098    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-data\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.410242 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.410164    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-credential-socket\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.410242 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.410235    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-certs\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.410459 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.410441    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-socket\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.410721 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.410700    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istiod-ca-cert\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.411998 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.411976    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-envoy\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.412354 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.412335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-podinfo\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.417839 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.417807    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-token\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.417941 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.417911    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r27m7\" (UniqueName: \"kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-kube-api-access-r27m7\") pod \"e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") " pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.518282 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.518249    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:45.596799 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.596710    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:45.735588 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.735458    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"]
Aug 04 03:44:45.735763 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:44:45.735738    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93bf88f2_0480_49e2_9477_d0ed1408cc1d.slice/crio-3d252ff40a35a690ce1f51b0baa97456d365b44468c70f7ce984ded358080b23 WatchSource:0}: Error finding container 3d252ff40a35a690ce1f51b0baa97456d365b44468c70f7ce984ded358080b23: Status 404 returned error can't find the container with id 3d252ff40a35a690ce1f51b0baa97456d365b44468c70f7ce984ded358080b23
Aug 04 03:44:45.738234 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.738206    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:44:45.738317 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.738275    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:44:45.738317 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:45.738303    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:44:46.351279 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:46.351234    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" event={"ID":"93bf88f2-0480-49e2-9477-d0ed1408cc1d","Type":"ContainerStarted","Data":"224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08"}
Aug 04 03:44:46.351806 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:46.351299    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" event={"ID":"93bf88f2-0480-49e2-9477-d0ed1408cc1d","Type":"ContainerStarted","Data":"3d252ff40a35a690ce1f51b0baa97456d365b44468c70f7ce984ded358080b23"}
Aug 04 03:44:46.371111 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:46.371058    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" podStartSLOduration=1.371035776 podStartE2EDuration="1.371035776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:44:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:44:46.369488855 +0000 UTC m=+2103.667629370" watchObservedRunningTime="2026-08-04 03:44:46.371035776 +0000 UTC m=+2103.669176263"
Aug 04 03:44:46.597258 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:46.597224    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:46.602299 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:46.602229    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:47.357017 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:47.356986    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:47.358098 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:47.358075    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:44:47.710371 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:47.710289    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:47.810685 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:47.810653    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:48.975953 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:48.975916    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:49.388522 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.388490    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:49.490801 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.490773    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:49.566556 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.566488    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:44:49.567291 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.567272    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"]
Aug 04 03:44:49.570657 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.570642    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:44:49.574450 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.574431    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-49bbb41b-dockercfg-jqz8l\""
Aug 04 03:44:49.574771 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.574756    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-49bbb41b\""
Aug 04 03:44:49.598027 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.594434    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"]
Aug 04 03:44:49.642047 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.641961    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3ff952e-6d24-4416-9306-35847020fb3e-plugins-config-volume\") pod \"payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg\" (UID: \"d3ff952e-6d24-4416-9306-35847020fb3e\") " pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:44:49.642047 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.642025    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfnxp\" (UniqueName: \"kubernetes.io/projected/d3ff952e-6d24-4416-9306-35847020fb3e-kube-api-access-zfnxp\") pod \"payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg\" (UID: \"d3ff952e-6d24-4416-9306-35847020fb3e\") " pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:44:49.742482 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.742446    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3ff952e-6d24-4416-9306-35847020fb3e-plugins-config-volume\") pod \"payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg\" (UID: \"d3ff952e-6d24-4416-9306-35847020fb3e\") " pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:44:49.742702 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.742517    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfnxp\" (UniqueName: \"kubernetes.io/projected/d3ff952e-6d24-4416-9306-35847020fb3e-kube-api-access-zfnxp\") pod \"payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg\" (UID: \"d3ff952e-6d24-4416-9306-35847020fb3e\") " pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:44:49.743166 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.743144    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3ff952e-6d24-4416-9306-35847020fb3e-plugins-config-volume\") pod \"payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg\" (UID: \"d3ff952e-6d24-4416-9306-35847020fb3e\") " pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:44:49.749745 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.749716    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfnxp\" (UniqueName: \"kubernetes.io/projected/d3ff952e-6d24-4416-9306-35847020fb3e-kube-api-access-zfnxp\") pod \"payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg\" (UID: \"d3ff952e-6d24-4416-9306-35847020fb3e\") " pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:44:49.880966 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:49.880937    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:44:50.217995 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:50.217972    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"]
Aug 04 03:44:50.220265 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:44:50.220238    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ff952e_6d24_4416_9306_35847020fb3e.slice/crio-ed916b94105f9a2b7ba5da14e20610938b75023829695d440d3123836ab21884 WatchSource:0}: Error finding container ed916b94105f9a2b7ba5da14e20610938b75023829695d440d3123836ab21884: Status 404 returned error can't find the container with id ed916b94105f9a2b7ba5da14e20610938b75023829695d440d3123836ab21884
Aug 04 03:44:50.373389 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:50.373352    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg" event={"ID":"d3ff952e-6d24-4416-9306-35847020fb3e","Type":"ContainerStarted","Data":"927e2e45d1820fa7aac0f1d90c084a232b2022f7eab9c03790add732fdf39a6d"}
Aug 04 03:44:50.373575 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:50.373414    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg" event={"ID":"d3ff952e-6d24-4416-9306-35847020fb3e","Type":"ContainerStarted","Data":"ed916b94105f9a2b7ba5da14e20610938b75023829695d440d3123836ab21884"}
Aug 04 03:44:50.373575 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:50.373455    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:44:50.389191 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:44:50.389147    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg" podStartSLOduration=1.389132578 podStartE2EDuration="1.389132578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:44:50.387556229 +0000 UTC m=+2107.685696731" watchObservedRunningTime="2026-08-04 03:44:50.389132578 +0000 UTC m=+2107.687273065"
Aug 04 03:45:00.140349 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.140288    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l"]
Aug 04 03:45:00.146168 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.146138    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn"]
Aug 04 03:45:00.150150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.150127    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l"]
Aug 04 03:45:00.150406 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.150392    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn"
Aug 04 03:45:00.150935 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.150918    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l"
Aug 04 03:45:00.152024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.151997    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn"]
Aug 04 03:45:00.153376 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.153356    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7lrf4\""
Aug 04 03:45:00.165649 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.165627    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn"
Aug 04 03:45:00.178894 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.178871    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l"
Aug 04 03:45:00.394037 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.394012    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l"]
Aug 04 03:45:00.396289 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:45:00.396257    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe1439f_fe14_4695_8023_feacf4ff4cef.slice/crio-570b20055be225f5eb1716647ad2ac2177eab3961ca9fd3aa25c45de47e3dbbd WatchSource:0}: Error finding container 570b20055be225f5eb1716647ad2ac2177eab3961ca9fd3aa25c45de47e3dbbd: Status 404 returned error can't find the container with id 570b20055be225f5eb1716647ad2ac2177eab3961ca9fd3aa25c45de47e3dbbd
Aug 04 03:45:00.422160 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.422130    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l" event={"ID":"bfe1439f-fe14-4695-8023-feacf4ff4cef","Type":"ContainerStarted","Data":"570b20055be225f5eb1716647ad2ac2177eab3961ca9fd3aa25c45de47e3dbbd"}
Aug 04 03:45:00.556571 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:00.555045    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn"]
Aug 04 03:45:00.558970 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:45:00.558935    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7940e64c_5b01_4660_8107_a2de58f69424.slice/crio-49a3ec06bbf471618f65e27a0f48c9a374d03229c7e528d7324e0d934c7431c3 WatchSource:0}: Error finding container 49a3ec06bbf471618f65e27a0f48c9a374d03229c7e528d7324e0d934c7431c3: Status 404 returned error can't find the container with id 49a3ec06bbf471618f65e27a0f48c9a374d03229c7e528d7324e0d934c7431c3
Aug 04 03:45:01.380570 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:01.380520    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:45:01.431565 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:01.430451    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn" event={"ID":"7940e64c-5b01-4660-8107-a2de58f69424","Type":"ContainerStarted","Data":"3bee1b9fa8cf70f24cbd672d181f2e9902821214740ff7dd6229e333571efd98"}
Aug 04 03:45:01.431565 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:01.430517    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn" event={"ID":"7940e64c-5b01-4660-8107-a2de58f69424","Type":"ContainerStarted","Data":"49a3ec06bbf471618f65e27a0f48c9a374d03229c7e528d7324e0d934c7431c3"}
Aug 04 03:45:01.433002 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:01.432976    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l" event={"ID":"bfe1439f-fe14-4695-8023-feacf4ff4cef","Type":"ContainerStarted","Data":"aba11956cdafa113fdb4b23af91c4008fdf8ea3aade5303d8bd967199710278c"}
Aug 04 03:45:01.446344 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:01.446289    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn" podStartSLOduration=1.4462719210000001 podStartE2EDuration="1.446271921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:45:01.445355893 +0000 UTC m=+2118.743496419" watchObservedRunningTime="2026-08-04 03:45:01.446271921 +0000 UTC m=+2118.744412409"
Aug 04 03:45:01.460527 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:01.460474    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l" podStartSLOduration=1.460458298 podStartE2EDuration="1.460458298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:45:01.459937981 +0000 UTC m=+2118.758078469" watchObservedRunningTime="2026-08-04 03:45:01.460458298 +0000 UTC m=+2118.758598786"
Aug 04 03:45:02.441338 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:02.441302    2559 generic.go:356] "Generic (PLEG): container finished" podID="bfe1439f-fe14-4695-8023-feacf4ff4cef" containerID="aba11956cdafa113fdb4b23af91c4008fdf8ea3aade5303d8bd967199710278c" exitCode=0
Aug 04 03:45:02.441788 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:02.441381    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l" event={"ID":"bfe1439f-fe14-4695-8023-feacf4ff4cef","Type":"ContainerDied","Data":"aba11956cdafa113fdb4b23af91c4008fdf8ea3aade5303d8bd967199710278c"}
Aug 04 03:45:02.442935 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:02.442909    2559 generic.go:356] "Generic (PLEG): container finished" podID="7940e64c-5b01-4660-8107-a2de58f69424" containerID="3bee1b9fa8cf70f24cbd672d181f2e9902821214740ff7dd6229e333571efd98" exitCode=0
Aug 04 03:45:02.443055 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:02.442997    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn" event={"ID":"7940e64c-5b01-4660-8107-a2de58f69424","Type":"ContainerDied","Data":"3bee1b9fa8cf70f24cbd672d181f2e9902821214740ff7dd6229e333571efd98"}
Aug 04 03:45:03.317154 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:03.317120    2559 patch_prober.go:28] interesting pod/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": read tcp 10.134.0.2:57212->10.134.0.44:15021: read: connection reset by peer" start-of-body=
Aug 04 03:45:03.317323 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:03.317188    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": read tcp 10.134.0.2:57212->10.134.0.44:15021: read: connection reset by peer"
Aug 04 03:45:03.449225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:03.449194    2559 generic.go:356] "Generic (PLEG): container finished" podID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerID="224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08" exitCode=0
Aug 04 03:45:03.449659 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:03.449275    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" event={"ID":"93bf88f2-0480-49e2-9477-d0ed1408cc1d","Type":"ContainerDied","Data":"224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08"}
Aug 04 03:45:03.450184 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:03.450021    2559 scope.go:117] "RemoveContainer" containerID="224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08"
Aug 04 03:45:03.451163 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:03.451135    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:03.451280 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:03.451205    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:03.451280 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:03.451239    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:03.610833 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:03.610810    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l"
Aug 04 03:45:03.613842 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:03.613814    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn"
Aug 04 03:45:04.459700 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:04.459662    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" event={"ID":"93bf88f2-0480-49e2-9477-d0ed1408cc1d","Type":"ContainerStarted","Data":"0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3"}
Aug 04 03:45:04.460904 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:04.460875    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l"
Aug 04 03:45:04.460904 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:04.460885    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763585-55m5l" event={"ID":"bfe1439f-fe14-4695-8023-feacf4ff4cef","Type":"ContainerDied","Data":"570b20055be225f5eb1716647ad2ac2177eab3961ca9fd3aa25c45de47e3dbbd"}
Aug 04 03:45:04.461125 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:04.460926    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="570b20055be225f5eb1716647ad2ac2177eab3961ca9fd3aa25c45de47e3dbbd"
Aug 04 03:45:04.462121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:04.462099    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn" event={"ID":"7940e64c-5b01-4660-8107-a2de58f69424","Type":"ContainerDied","Data":"49a3ec06bbf471618f65e27a0f48c9a374d03229c7e528d7324e0d934c7431c3"}
Aug 04 03:45:04.462225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:04.462121    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn"
Aug 04 03:45:04.462225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:04.462132    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49a3ec06bbf471618f65e27a0f48c9a374d03229c7e528d7324e0d934c7431c3"
Aug 04 03:45:04.597374 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:04.597337    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:45:04.597613 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:04.597591    2559 patch_prober.go:28] interesting pod/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Aug 04 03:45:04.597671 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:04.597641    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Aug 04 03:45:05.597138 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:05.597102    2559 patch_prober.go:28] interesting pod/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Aug 04 03:45:05.597515 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:05.597170    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Aug 04 03:45:06.597499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:06.597465    2559 patch_prober.go:28] interesting pod/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Aug 04 03:45:06.598014 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:06.597563    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Aug 04 03:45:07.597988 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:07.597948    2559 patch_prober.go:28] interesting pod/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Aug 04 03:45:07.598370 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:07.598004    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Aug 04 03:45:08.600793 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:08.600762    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:45:08.601158 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:08.601032    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:45:08.601820 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:08.601805    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:45:13.184168 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.184134    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"]
Aug 04 03:45:13.184574 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.184486    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfe1439f-fe14-4695-8023-feacf4ff4cef" containerName="cleanup"
Aug 04 03:45:13.184574 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.184496    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfe1439f-fe14-4695-8023-feacf4ff4cef" containerName="cleanup"
Aug 04 03:45:13.184574 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.184515    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7940e64c-5b01-4660-8107-a2de58f69424" containerName="cleanup"
Aug 04 03:45:13.184574 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.184521    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7940e64c-5b01-4660-8107-a2de58f69424" containerName="cleanup"
Aug 04 03:45:13.184723 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.184588    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7940e64c-5b01-4660-8107-a2de58f69424" containerName="cleanup"
Aug 04 03:45:13.184723 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.184599    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfe1439f-fe14-4695-8023-feacf4ff4cef" containerName="cleanup"
Aug 04 03:45:13.187749 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.187731    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"
Aug 04 03:45:13.189815 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.189797    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 03:45:13.189815 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.189808    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7lrf4\""
Aug 04 03:45:13.195287 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.195202    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"]
Aug 04 03:45:13.266384 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.266351    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c432a30-73e6-4b9a-9f40-bc1bb6f553b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r\" (UID: \"1c432a30-73e6-4b9a-9f40-bc1bb6f553b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"
Aug 04 03:45:13.367813 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.367779    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c432a30-73e6-4b9a-9f40-bc1bb6f553b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r\" (UID: \"1c432a30-73e6-4b9a-9f40-bc1bb6f553b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"
Aug 04 03:45:13.368409 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.368388    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c432a30-73e6-4b9a-9f40-bc1bb6f553b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r\" (UID: \"1c432a30-73e6-4b9a-9f40-bc1bb6f553b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"
Aug 04 03:45:13.499384 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.499354    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"
Aug 04 03:45:13.641463 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:13.641431    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"]
Aug 04 03:45:13.643067 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:45:13.643041    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c432a30_73e6_4b9a_9f40_bc1bb6f553b7.slice/crio-354d1fb152538052c207a7c922650a7334bb2fd3032d1a2963429b9e76b015c3 WatchSource:0}: Error finding container 354d1fb152538052c207a7c922650a7334bb2fd3032d1a2963429b9e76b015c3: Status 404 returned error can't find the container with id 354d1fb152538052c207a7c922650a7334bb2fd3032d1a2963429b9e76b015c3
Aug 04 03:45:14.516973 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:14.516941    2559 generic.go:356] "Generic (PLEG): container finished" podID="1c432a30-73e6-4b9a-9f40-bc1bb6f553b7" containerID="168cae2ca18b04ff3bb2bb1f32f1a3b494286061649de106f2590a2e4bce2b10" exitCode=0
Aug 04 03:45:14.517377 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:14.517004    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r" event={"ID":"1c432a30-73e6-4b9a-9f40-bc1bb6f553b7","Type":"ContainerDied","Data":"168cae2ca18b04ff3bb2bb1f32f1a3b494286061649de106f2590a2e4bce2b10"}
Aug 04 03:45:14.517377 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:14.517049    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r" event={"ID":"1c432a30-73e6-4b9a-9f40-bc1bb6f553b7","Type":"ContainerStarted","Data":"354d1fb152538052c207a7c922650a7334bb2fd3032d1a2963429b9e76b015c3"}
Aug 04 03:45:15.656355 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:15.656331    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"
Aug 04 03:45:15.787058 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:15.786977    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c432a30-73e6-4b9a-9f40-bc1bb6f553b7-maas-api-service-ca\") pod \"1c432a30-73e6-4b9a-9f40-bc1bb6f553b7\" (UID: \"1c432a30-73e6-4b9a-9f40-bc1bb6f553b7\") "
Aug 04 03:45:15.787330 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:15.787307    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c432a30-73e6-4b9a-9f40-bc1bb6f553b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1c432a30-73e6-4b9a-9f40-bc1bb6f553b7" (UID: "1c432a30-73e6-4b9a-9f40-bc1bb6f553b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:45:15.887704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:15.887670    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c432a30-73e6-4b9a-9f40-bc1bb6f553b7-maas-api-service-ca\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:16.528385 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:16.528355    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"
Aug 04 03:45:16.528582 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:16.528391    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r" event={"ID":"1c432a30-73e6-4b9a-9f40-bc1bb6f553b7","Type":"ContainerDied","Data":"354d1fb152538052c207a7c922650a7334bb2fd3032d1a2963429b9e76b015c3"}
Aug 04 03:45:16.528582 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:16.528435    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="354d1fb152538052c207a7c922650a7334bb2fd3032d1a2963429b9e76b015c3"
Aug 04 03:45:19.263720 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.263686    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn"]
Aug 04 03:45:19.265499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.265479    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-49bbb41b-29763585-zpmzn"]
Aug 04 03:45:19.292482 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.292443    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"]
Aug 04 03:45:19.292784 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.292755    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg" podUID="d3ff952e-6d24-4416-9306-35847020fb3e" containerName="payload-processing" containerID="cri-o://927e2e45d1820fa7aac0f1d90c084a232b2022f7eab9c03790add732fdf39a6d" gracePeriod=30
Aug 04 03:45:19.327825 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.327791    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:19.419395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.419368    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"]
Aug 04 03:45:19.422309 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.422282    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:19.422427 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.422319    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-49bbb41b-mkx4r"]
Aug 04 03:45:19.478799 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.475946    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:19.547467 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.547408    2559 generic.go:356] "Generic (PLEG): container finished" podID="d3ff952e-6d24-4416-9306-35847020fb3e" containerID="927e2e45d1820fa7aac0f1d90c084a232b2022f7eab9c03790add732fdf39a6d" exitCode=0
Aug 04 03:45:19.547962 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.547918    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg" event={"ID":"d3ff952e-6d24-4416-9306-35847020fb3e","Type":"ContainerDied","Data":"927e2e45d1820fa7aac0f1d90c084a232b2022f7eab9c03790add732fdf39a6d"}
Aug 04 03:45:19.548091 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.548019    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg" event={"ID":"d3ff952e-6d24-4416-9306-35847020fb3e","Type":"ContainerDied","Data":"ed916b94105f9a2b7ba5da14e20610938b75023829695d440d3123836ab21884"}
Aug 04 03:45:19.548091 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.548050    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed916b94105f9a2b7ba5da14e20610938b75023829695d440d3123836ab21884"
Aug 04 03:45:19.552557 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.552523    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:45:19.557916 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.557895    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"]
Aug 04 03:45:19.558074 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.558054    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy" containerID="cri-o://0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3" gracePeriod=30
Aug 04 03:45:19.609389 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.609352    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:19.622912 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.622886    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3ff952e-6d24-4416-9306-35847020fb3e-plugins-config-volume\") pod \"d3ff952e-6d24-4416-9306-35847020fb3e\" (UID: \"d3ff952e-6d24-4416-9306-35847020fb3e\") "
Aug 04 03:45:19.623063 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.622926    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfnxp\" (UniqueName: \"kubernetes.io/projected/d3ff952e-6d24-4416-9306-35847020fb3e-kube-api-access-zfnxp\") pod \"d3ff952e-6d24-4416-9306-35847020fb3e\" (UID: \"d3ff952e-6d24-4416-9306-35847020fb3e\") "
Aug 04 03:45:19.623281 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.623254    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3ff952e-6d24-4416-9306-35847020fb3e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d3ff952e-6d24-4416-9306-35847020fb3e" (UID: "d3ff952e-6d24-4416-9306-35847020fb3e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:45:19.625128 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.625107    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3ff952e-6d24-4416-9306-35847020fb3e-kube-api-access-zfnxp" (OuterVolumeSpecName: "kube-api-access-zfnxp") pod "d3ff952e-6d24-4416-9306-35847020fb3e" (UID: "d3ff952e-6d24-4416-9306-35847020fb3e"). InnerVolumeSpecName "kube-api-access-zfnxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:45:19.723958 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.723918    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3ff952e-6d24-4416-9306-35847020fb3e-plugins-config-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:19.723958 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.723950    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfnxp\" (UniqueName: \"kubernetes.io/projected/d3ff952e-6d24-4416-9306-35847020fb3e-kube-api-access-zfnxp\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:19.785913 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:19.785832    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:20.017127 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.017079    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:20.118329 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.118225    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:20.245270 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.245236    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:20.496075 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.496038    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"]
Aug 04 03:45:20.496443 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.496381    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3ff952e-6d24-4416-9306-35847020fb3e" containerName="payload-processing"
Aug 04 03:45:20.496443 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.496392    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3ff952e-6d24-4416-9306-35847020fb3e" containerName="payload-processing"
Aug 04 03:45:20.496443 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.496411    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c432a30-73e6-4b9a-9f40-bc1bb6f553b7" containerName="revoke-keys"
Aug 04 03:45:20.496443 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.496416    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c432a30-73e6-4b9a-9f40-bc1bb6f553b7" containerName="revoke-keys"
Aug 04 03:45:20.496588 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.496475    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c432a30-73e6-4b9a-9f40-bc1bb6f553b7" containerName="revoke-keys"
Aug 04 03:45:20.496588 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.496482    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3ff952e-6d24-4416-9306-35847020fb3e" containerName="payload-processing"
Aug 04 03:45:20.499505 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.499487    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.502383 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.502363    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-822464dd-gw-openshift-default-dockercfg-x2dp7\""
Aug 04 03:45:20.521367 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.521336    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"]
Aug 04 03:45:20.531085 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.531046    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:20.552367 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.552339    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"
Aug 04 03:45:20.579485 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.579450    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"]
Aug 04 03:45:20.582335 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.582305    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-49bbb41b-55cf8dbb64-xb7vg"]
Aug 04 03:45:20.632429 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.632392    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istiod-ca-cert\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.632623 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.632466    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-certs\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.632623 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.632526    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-credential-socket\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.632623 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.632581    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-socket\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.632623 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.632607    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-envoy\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.632781 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.632662    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-token\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.632781 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.632692    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm9p5\" (UniqueName: \"kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-kube-api-access-hm9p5\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.632781 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.632751    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-data\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.632781 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.632775    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-podinfo\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.733908 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.733872    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-certs\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.733920    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-credential-socket\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.733942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-socket\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.733957    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-envoy\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.734005    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-token\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.734029    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm9p5\" (UniqueName: \"kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-kube-api-access-hm9p5\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.734063    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-data\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.734089    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-podinfo\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.734120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istiod-ca-cert\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734915 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.734352    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-socket\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734915 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.734399    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-credential-socket\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734915 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.734733    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-certs\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.734915 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.734743    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-data\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.735107 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.735088    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istiod-ca-cert\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.737142 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.737110    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-envoy\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.737142 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.737125    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-podinfo\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.741532 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.741504    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-token\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.741658 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.741604    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm9p5\" (UniqueName: \"kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-kube-api-access-hm9p5\") pod \"e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") " pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.754493 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.754427    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:20.809634 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.809597    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:20.947950 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.947925    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"]
Aug 04 03:45:20.949884 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:45:20.949849    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad17e8b_8f62_42bb_b072_c098c284b6fa.slice/crio-919507479b08d8da62afabddbe4e2f4a7502b13903e7e4b7a25d9b3cba27a47d WatchSource:0}: Error finding container 919507479b08d8da62afabddbe4e2f4a7502b13903e7e4b7a25d9b3cba27a47d: Status 404 returned error can't find the container with id 919507479b08d8da62afabddbe4e2f4a7502b13903e7e4b7a25d9b3cba27a47d
Aug 04 03:45:20.952165 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.952119    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:20.952252 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.952220    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:20.952287 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:20.952266    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:21.238138 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:21.238107    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c432a30-73e6-4b9a-9f40-bc1bb6f553b7" path="/var/lib/kubelet/pods/1c432a30-73e6-4b9a-9f40-bc1bb6f553b7/volumes"
Aug 04 03:45:21.238393 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:21.238380    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7940e64c-5b01-4660-8107-a2de58f69424" path="/var/lib/kubelet/pods/7940e64c-5b01-4660-8107-a2de58f69424/volumes"
Aug 04 03:45:21.238676 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:21.238664    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3ff952e-6d24-4416-9306-35847020fb3e" path="/var/lib/kubelet/pods/d3ff952e-6d24-4416-9306-35847020fb3e/volumes"
Aug 04 03:45:21.558953 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:21.558909    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" event={"ID":"7ad17e8b-8f62-42bb-b072-c098c284b6fa","Type":"ContainerStarted","Data":"74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f"}
Aug 04 03:45:21.559362 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:21.558968    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" event={"ID":"7ad17e8b-8f62-42bb-b072-c098c284b6fa","Type":"ContainerStarted","Data":"919507479b08d8da62afabddbe4e2f4a7502b13903e7e4b7a25d9b3cba27a47d"}
Aug 04 03:45:21.578180 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:21.578133    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" podStartSLOduration=1.578118897 podStartE2EDuration="1.578118897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:45:21.576740315 +0000 UTC m=+2138.874880829" watchObservedRunningTime="2026-08-04 03:45:21.578118897 +0000 UTC m=+2138.876259385"
Aug 04 03:45:21.810250 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:21.810163    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:22.815205 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:22.815174    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:23.043672 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.043636    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:23.146575 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.146469    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:23.248232 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.248202    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:23.570149 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.570119    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:23.571233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.571211    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:23.601162 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.601107    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:23.603156 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.603125    2559 patch_prober.go:28] interesting pod/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 03:45:23.603378 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.603350    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 03:45:23.693317 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.693285    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:23.755381 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.755344    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"]
Aug 04 03:45:23.759945 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.759927    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:23.762637 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.762608    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-822464dd\""
Aug 04 03:45:23.762775 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.762751    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-822464dd-dockercfg-8jhzw\""
Aug 04 03:45:23.769155 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.769131    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"]
Aug 04 03:45:23.863565 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.863449    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6jxx\" (UniqueName: \"kubernetes.io/projected/ac3746fa-27e3-4a26-858b-6bb35d846335-kube-api-access-t6jxx\") pod \"payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf\" (UID: \"ac3746fa-27e3-4a26-858b-6bb35d846335\") " pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:23.863565 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.863530    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac3746fa-27e3-4a26-858b-6bb35d846335-plugins-config-volume\") pod \"payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf\" (UID: \"ac3746fa-27e3-4a26-858b-6bb35d846335\") " pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:23.964825 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.964789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6jxx\" (UniqueName: \"kubernetes.io/projected/ac3746fa-27e3-4a26-858b-6bb35d846335-kube-api-access-t6jxx\") pod \"payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf\" (UID: \"ac3746fa-27e3-4a26-858b-6bb35d846335\") " pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:23.965018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.964860    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac3746fa-27e3-4a26-858b-6bb35d846335-plugins-config-volume\") pod \"payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf\" (UID: \"ac3746fa-27e3-4a26-858b-6bb35d846335\") " pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:23.965601 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.965579    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac3746fa-27e3-4a26-858b-6bb35d846335-plugins-config-volume\") pod \"payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf\" (UID: \"ac3746fa-27e3-4a26-858b-6bb35d846335\") " pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:23.972472 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:23.972445    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6jxx\" (UniqueName: \"kubernetes.io/projected/ac3746fa-27e3-4a26-858b-6bb35d846335-kube-api-access-t6jxx\") pod \"payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf\" (UID: \"ac3746fa-27e3-4a26-858b-6bb35d846335\") " pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:24.071708 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.071670    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:24.413869 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.413846    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"]
Aug 04 03:45:24.415645 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:45:24.415613    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac3746fa_27e3_4a26_858b_6bb35d846335.slice/crio-c95d84979e4cb7de1c374310fe46ccab14345941ccfc2e4e89a639c8e4b067a2 WatchSource:0}: Error finding container c95d84979e4cb7de1c374310fe46ccab14345941ccfc2e4e89a639c8e4b067a2: Status 404 returned error can't find the container with id c95d84979e4cb7de1c374310fe46ccab14345941ccfc2e4e89a639c8e4b067a2
Aug 04 03:45:24.575840 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.575812    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf" event={"ID":"ac3746fa-27e3-4a26-858b-6bb35d846335","Type":"ContainerStarted","Data":"0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a"}
Aug 04 03:45:24.575970 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.575863    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf" event={"ID":"ac3746fa-27e3-4a26-858b-6bb35d846335","Type":"ContainerStarted","Data":"c95d84979e4cb7de1c374310fe46ccab14345941ccfc2e4e89a639c8e4b067a2"}
Aug 04 03:45:24.576027 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.576010    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:24.591629 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.591587    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf" podStartSLOduration=1.591570706 podStartE2EDuration="1.591570706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:45:24.590126819 +0000 UTC m=+2141.888267307" watchObservedRunningTime="2026-08-04 03:45:24.591570706 +0000 UTC m=+2141.889711194"
Aug 04 03:45:24.801234 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.801208    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:45:24.974044 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.973950    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-token\") pod \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") "
Aug 04 03:45:24.974044 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974031    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-socket\") pod \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") "
Aug 04 03:45:24.974537 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974066    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istiod-ca-cert\") pod \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") "
Aug 04 03:45:24.974537 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974097    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-podinfo\") pod \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") "
Aug 04 03:45:24.974537 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974182    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r27m7\" (UniqueName: \"kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-kube-api-access-r27m7\") pod \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") "
Aug 04 03:45:24.974537 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974219    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-envoy\") pod \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") "
Aug 04 03:45:24.974537 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974249    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-certs\") pod \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") "
Aug 04 03:45:24.974537 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974284    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-credential-socket\") pod \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") "
Aug 04 03:45:24.974537 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974309    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-data\") pod \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\" (UID: \"93bf88f2-0480-49e2-9477-d0ed1408cc1d\") "
Aug 04 03:45:24.974537 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974368    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93bf88f2-0480-49e2-9477-d0ed1408cc1d" (UID: "93bf88f2-0480-49e2-9477-d0ed1408cc1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:45:24.974537 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974430    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93bf88f2-0480-49e2-9477-d0ed1408cc1d" (UID: "93bf88f2-0480-49e2-9477-d0ed1408cc1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:45:24.975039 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974629    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:24.975039 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974653    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:24.975039 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974785    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93bf88f2-0480-49e2-9477-d0ed1408cc1d" (UID: "93bf88f2-0480-49e2-9477-d0ed1408cc1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:45:24.975039 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.974902    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93bf88f2-0480-49e2-9477-d0ed1408cc1d" (UID: "93bf88f2-0480-49e2-9477-d0ed1408cc1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:45:24.975331 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.975265    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-data" (OuterVolumeSpecName: "istio-data") pod "93bf88f2-0480-49e2-9477-d0ed1408cc1d" (UID: "93bf88f2-0480-49e2-9477-d0ed1408cc1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:45:24.976736 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.976705    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-kube-api-access-r27m7" (OuterVolumeSpecName: "kube-api-access-r27m7") pod "93bf88f2-0480-49e2-9477-d0ed1408cc1d" (UID: "93bf88f2-0480-49e2-9477-d0ed1408cc1d"). InnerVolumeSpecName "kube-api-access-r27m7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:45:24.976736 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.976733    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93bf88f2-0480-49e2-9477-d0ed1408cc1d" (UID: "93bf88f2-0480-49e2-9477-d0ed1408cc1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 03:45:24.976904 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.976794    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93bf88f2-0480-49e2-9477-d0ed1408cc1d" (UID: "93bf88f2-0480-49e2-9477-d0ed1408cc1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:45:24.977259 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:24.977245    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-token" (OuterVolumeSpecName: "istio-token") pod "93bf88f2-0480-49e2-9477-d0ed1408cc1d" (UID: "93bf88f2-0480-49e2-9477-d0ed1408cc1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:45:25.075896 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.075854    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:25.075896 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.075891    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:25.075896 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.075900    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:25.075896 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.075908    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:25.076165 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.075918    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r27m7\" (UniqueName: \"kubernetes.io/projected/93bf88f2-0480-49e2-9477-d0ed1408cc1d-kube-api-access-r27m7\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:25.076165 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.075926    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:25.076165 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.075936    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93bf88f2-0480-49e2-9477-d0ed1408cc1d-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:25.581713 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.581628    2559 generic.go:356] "Generic (PLEG): container finished" podID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerID="0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3" exitCode=0
Aug 04 03:45:25.581713 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.581677    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"
Aug 04 03:45:25.581952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.581718    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" event={"ID":"93bf88f2-0480-49e2-9477-d0ed1408cc1d","Type":"ContainerDied","Data":"0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3"}
Aug 04 03:45:25.581952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.581779    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n" event={"ID":"93bf88f2-0480-49e2-9477-d0ed1408cc1d","Type":"ContainerDied","Data":"3d252ff40a35a690ce1f51b0baa97456d365b44468c70f7ce984ded358080b23"}
Aug 04 03:45:25.581952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.581808    2559 scope.go:117] "RemoveContainer" containerID="0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3"
Aug 04 03:45:25.591305 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.591070    2559 scope.go:117] "RemoveContainer" containerID="224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08"
Aug 04 03:45:25.599943 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.599914    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"]
Aug 04 03:45:25.600806 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.600790    2559 scope.go:117] "RemoveContainer" containerID="0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3"
Aug 04 03:45:25.601077 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:45:25.601055    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3\": container with ID starting with 0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3 not found: ID does not exist" containerID="0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3"
Aug 04 03:45:25.601115 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.601091    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3"} err="failed to get container status \"0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3\": rpc error: code = NotFound desc = could not find container \"0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3\": container with ID starting with 0014a0426ca97dba295491f6cd7bea3dfad01385a2014179af75029ff8ee47d3 not found: ID does not exist"
Aug 04 03:45:25.601115 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.601111    2559 scope.go:117] "RemoveContainer" containerID="224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08"
Aug 04 03:45:25.601356 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:45:25.601333    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08\": container with ID starting with 224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08 not found: ID does not exist" containerID="224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08"
Aug 04 03:45:25.601433 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.601366    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08"} err="failed to get container status \"224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08\": rpc error: code = NotFound desc = could not find container \"224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08\": container with ID starting with 224804b69e76d7e974292296474607c9c349db397f1a746bd864364c3be9cd08 not found: ID does not exist"
Aug 04 03:45:25.604139 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:25.604115    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-49bbb41b-openshift-default-58f995966d-gzs2n"]
Aug 04 03:45:27.239026 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:27.238992    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" path="/var/lib/kubelet/pods/93bf88f2-0480-49e2-9477-d0ed1408cc1d/volumes"
Aug 04 03:45:35.585123 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:35.585076    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:37.645301 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:37.645205    2559 generic.go:356] "Generic (PLEG): container finished" podID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerID="74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f" exitCode=0
Aug 04 03:45:37.645301 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:37.645282    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" event={"ID":"7ad17e8b-8f62-42bb-b072-c098c284b6fa","Type":"ContainerDied","Data":"74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f"}
Aug 04 03:45:37.645906 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:37.645771    2559 scope.go:117] "RemoveContainer" containerID="74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f"
Aug 04 03:45:37.646817 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:37.646787    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:37.646922 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:37.646857    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:37.646922 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:37.646887    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:38.570801 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:38.570759    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:38.663158 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:38.663119    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" event={"ID":"7ad17e8b-8f62-42bb-b072-c098c284b6fa","Type":"ContainerStarted","Data":"659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176"}
Aug 04 03:45:38.810530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:38.810487    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:38.811734 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:38.811707    2559 patch_prober.go:28] interesting pod/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Aug 04 03:45:38.811882 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:38.811779    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Aug 04 03:45:39.810914 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:39.810871    2559 patch_prober.go:28] interesting pod/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Aug 04 03:45:39.811333 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:39.810935    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Aug 04 03:45:40.810018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:40.809981    2559 patch_prober.go:28] interesting pod/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Aug 04 03:45:40.810200 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:40.810040    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Aug 04 03:45:41.810863 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:41.810824    2559 patch_prober.go:28] interesting pod/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Aug 04 03:45:41.811243 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:41.810885    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Aug 04 03:45:42.814074 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:42.814041    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:42.814438 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:42.814320    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:42.815082 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:42.815067    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:47.976463 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:47.976425    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"]
Aug 04 03:45:47.976832 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:47.976795    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy"
Aug 04 03:45:47.976832 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:47.976806    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy"
Aug 04 03:45:47.976832 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:47.976813    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy"
Aug 04 03:45:47.976832 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:47.976819    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy"
Aug 04 03:45:47.976976 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:47.976884    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy"
Aug 04 03:45:47.976976 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:47.976891    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="93bf88f2-0480-49e2-9477-d0ed1408cc1d" containerName="istio-proxy"
Aug 04 03:45:47.978869 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:47.978850    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"
Aug 04 03:45:47.981280 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:47.981259    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 03:45:47.981365 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:47.981259    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7lrf4\""
Aug 04 03:45:47.987735 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:47.987714    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"]
Aug 04 03:45:48.101614 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:48.101585    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f269bf-5bc0-49cf-97ec-5bee4ebb5149-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9\" (UID: \"40f269bf-5bc0-49cf-97ec-5bee4ebb5149\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"
Aug 04 03:45:48.202252 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:48.202219    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f269bf-5bc0-49cf-97ec-5bee4ebb5149-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9\" (UID: \"40f269bf-5bc0-49cf-97ec-5bee4ebb5149\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"
Aug 04 03:45:48.202923 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:48.202898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f269bf-5bc0-49cf-97ec-5bee4ebb5149-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9\" (UID: \"40f269bf-5bc0-49cf-97ec-5bee4ebb5149\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"
Aug 04 03:45:48.291925 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:48.291879    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"
Aug 04 03:45:48.421517 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:48.421493    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"]
Aug 04 03:45:48.423684 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:45:48.423657    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40f269bf_5bc0_49cf_97ec_5bee4ebb5149.slice/crio-9ba3402d2a4b2d9cf295798b041cc0e70e6485fb8982533a9509a2e875f0efff WatchSource:0}: Error finding container 9ba3402d2a4b2d9cf295798b041cc0e70e6485fb8982533a9509a2e875f0efff: Status 404 returned error can't find the container with id 9ba3402d2a4b2d9cf295798b041cc0e70e6485fb8982533a9509a2e875f0efff
Aug 04 03:45:48.712808 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:48.712771    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9" event={"ID":"40f269bf-5bc0-49cf-97ec-5bee4ebb5149","Type":"ContainerStarted","Data":"9ba3402d2a4b2d9cf295798b041cc0e70e6485fb8982533a9509a2e875f0efff"}
Aug 04 03:45:49.718825 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:49.718788    2559 generic.go:356] "Generic (PLEG): container finished" podID="40f269bf-5bc0-49cf-97ec-5bee4ebb5149" containerID="845fee39c2a5c40e3446e7322428cdf397eba5d93a944e8884d6325872689fef" exitCode=0
Aug 04 03:45:49.719239 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:49.718868    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9" event={"ID":"40f269bf-5bc0-49cf-97ec-5bee4ebb5149","Type":"ContainerDied","Data":"845fee39c2a5c40e3446e7322428cdf397eba5d93a944e8884d6325872689fef"}
Aug 04 03:45:50.855353 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:50.855325    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"
Aug 04 03:45:50.926160 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:50.926128    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f269bf-5bc0-49cf-97ec-5bee4ebb5149-maas-api-service-ca\") pod \"40f269bf-5bc0-49cf-97ec-5bee4ebb5149\" (UID: \"40f269bf-5bc0-49cf-97ec-5bee4ebb5149\") "
Aug 04 03:45:50.926537 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:50.926514    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40f269bf-5bc0-49cf-97ec-5bee4ebb5149-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "40f269bf-5bc0-49cf-97ec-5bee4ebb5149" (UID: "40f269bf-5bc0-49cf-97ec-5bee4ebb5149"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:45:51.027784 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:51.027749    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40f269bf-5bc0-49cf-97ec-5bee4ebb5149-maas-api-service-ca\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:51.729592 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:51.729561    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"
Aug 04 03:45:51.729592 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:51.729581    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9" event={"ID":"40f269bf-5bc0-49cf-97ec-5bee4ebb5149","Type":"ContainerDied","Data":"9ba3402d2a4b2d9cf295798b041cc0e70e6485fb8982533a9509a2e875f0efff"}
Aug 04 03:45:51.729803 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:51.729633    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ba3402d2a4b2d9cf295798b041cc0e70e6485fb8982533a9509a2e875f0efff"
Aug 04 03:45:53.636341 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.636302    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:53.655872 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.655837    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"]
Aug 04 03:45:53.656140 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.656107    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf" podUID="ac3746fa-27e3-4a26-858b-6bb35d846335" containerName="payload-processing" containerID="cri-o://0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a" gracePeriod=30
Aug 04 03:45:53.700200 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.700150    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:53.780293 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.780263    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"]
Aug 04 03:45:53.782646 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.782615    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:53.783190 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.783165    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-822464dd-4c9w9"]
Aug 04 03:45:53.837964 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.837930    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:53.913704 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.913677    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"]
Aug 04 03:45:53.913897 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.913874    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy" containerID="cri-o://659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176" gracePeriod=30
Aug 04 03:45:53.930195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.930144    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:53.932156 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:53.932123    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:54.058239 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.058204    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6jxx\" (UniqueName: \"kubernetes.io/projected/ac3746fa-27e3-4a26-858b-6bb35d846335-kube-api-access-t6jxx\") pod \"ac3746fa-27e3-4a26-858b-6bb35d846335\" (UID: \"ac3746fa-27e3-4a26-858b-6bb35d846335\") "
Aug 04 03:45:54.058384 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.058331    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac3746fa-27e3-4a26-858b-6bb35d846335-plugins-config-volume\") pod \"ac3746fa-27e3-4a26-858b-6bb35d846335\" (UID: \"ac3746fa-27e3-4a26-858b-6bb35d846335\") "
Aug 04 03:45:54.058737 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.058709    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac3746fa-27e3-4a26-858b-6bb35d846335-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac3746fa-27e3-4a26-858b-6bb35d846335" (UID: "ac3746fa-27e3-4a26-858b-6bb35d846335"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:45:54.060354 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.060330    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac3746fa-27e3-4a26-858b-6bb35d846335-kube-api-access-t6jxx" (OuterVolumeSpecName: "kube-api-access-t6jxx") pod "ac3746fa-27e3-4a26-858b-6bb35d846335" (UID: "ac3746fa-27e3-4a26-858b-6bb35d846335"). InnerVolumeSpecName "kube-api-access-t6jxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:45:54.145953 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.145911    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:54.159217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.159146    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6jxx\" (UniqueName: \"kubernetes.io/projected/ac3746fa-27e3-4a26-858b-6bb35d846335-kube-api-access-t6jxx\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:54.159217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.159180    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac3746fa-27e3-4a26-858b-6bb35d846335-plugins-config-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:54.361148 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.361110    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:54.628939 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.628902    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"]
Aug 04 03:45:54.629452 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.629430    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac3746fa-27e3-4a26-858b-6bb35d846335" containerName="payload-processing"
Aug 04 03:45:54.629567 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.629455    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac3746fa-27e3-4a26-858b-6bb35d846335" containerName="payload-processing"
Aug 04 03:45:54.629567 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.629507    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40f269bf-5bc0-49cf-97ec-5bee4ebb5149" containerName="revoke-keys"
Aug 04 03:45:54.629567 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.629517    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="40f269bf-5bc0-49cf-97ec-5bee4ebb5149" containerName="revoke-keys"
Aug 04 03:45:54.629728 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.629623    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="40f269bf-5bc0-49cf-97ec-5bee4ebb5149" containerName="revoke-keys"
Aug 04 03:45:54.629728 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.629635    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac3746fa-27e3-4a26-858b-6bb35d846335" containerName="payload-processing"
Aug 04 03:45:54.632076 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.632052    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.634826 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.634805    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9e9f9650-openshift-default-dockercfg-5mtbx\""
Aug 04 03:45:54.644268 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.644246    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"]
Aug 04 03:45:54.670878 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.670854    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:54.747293 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.747252    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:54.750221 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.750194    2559 generic.go:356] "Generic (PLEG): container finished" podID="ac3746fa-27e3-4a26-858b-6bb35d846335" containerID="0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a" exitCode=0
Aug 04 03:45:54.750341 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.750247    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"
Aug 04 03:45:54.750341 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.750281    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf" event={"ID":"ac3746fa-27e3-4a26-858b-6bb35d846335","Type":"ContainerDied","Data":"0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a"}
Aug 04 03:45:54.750416 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.750358    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf" event={"ID":"ac3746fa-27e3-4a26-858b-6bb35d846335","Type":"ContainerDied","Data":"c95d84979e4cb7de1c374310fe46ccab14345941ccfc2e4e89a639c8e4b067a2"}
Aug 04 03:45:54.750416 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.750387    2559 scope.go:117] "RemoveContainer" containerID="0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a"
Aug 04 03:45:54.758774 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.758747    2559 scope.go:117] "RemoveContainer" containerID="0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a"
Aug 04 03:45:54.759099 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:45:54.759078    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a\": container with ID starting with 0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a not found: ID does not exist" containerID="0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a"
Aug 04 03:45:54.759153 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.759112    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a"} err="failed to get container status \"0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a\": rpc error: code = NotFound desc = could not find container \"0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a\": container with ID starting with 0364043c7a5a1fc45e76d09c30a5557004cf31f0f57f1b2ecc43de562c526d7a not found: ID does not exist"
Aug 04 03:45:54.765643 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.765622    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-certs\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.765700 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.765650    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr756\" (UniqueName: \"kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-kube-api-access-gr756\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.765700 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.765691    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-socket\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.765766 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.765708    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-envoy\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.765766 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.765746    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-istio-token\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.765833 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.765778    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/292573e8-6c48-4917-af9e-ec78c64d104c-istiod-ca-cert\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.765833 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.765814    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-credential-socket\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.765899 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.765862    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-data\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.765940 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.765898    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/292573e8-6c48-4917-af9e-ec78c64d104c-istio-podinfo\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.770748 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.770714    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"]
Aug 04 03:45:54.776455 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.776433    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-822464dd-5d8cc99565-b6hzf"]
Aug 04 03:45:54.866889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.866849    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-socket\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.866889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.866888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-envoy\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867095 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.866916    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-istio-token\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867133 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.867101    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/292573e8-6c48-4917-af9e-ec78c64d104c-istiod-ca-cert\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867198 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.867181    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-credential-socket\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867248 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.867232    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-data\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867317 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.867277    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/292573e8-6c48-4917-af9e-ec78c64d104c-istio-podinfo\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867317 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.867298    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-socket\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867426 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.867359    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-certs\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867426 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.867386    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr756\" (UniqueName: \"kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-kube-api-access-gr756\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867593 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.867571    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-credential-socket\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867687 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.867634    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-data\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867798 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.867774    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-certs\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.867906 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.867885    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/292573e8-6c48-4917-af9e-ec78c64d104c-istiod-ca-cert\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.869352 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.869332    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-envoy\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.869592 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.869578    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/292573e8-6c48-4917-af9e-ec78c64d104c-istio-podinfo\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.875097 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.875072    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-istio-token\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.875295 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.875271    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr756\" (UniqueName: \"kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-kube-api-access-gr756\") pod \"e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") " pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:54.944102 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:54.944012    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:55.098115 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:55.098080    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"]
Aug 04 03:45:55.099608 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:45:55.099578    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod292573e8_6c48_4917_af9e_ec78c64d104c.slice/crio-3400ea7efe246280879a61ccf340748f7cd8c2b9ed427ec93c5adcdda66eda72 WatchSource:0}: Error finding container 3400ea7efe246280879a61ccf340748f7cd8c2b9ed427ec93c5adcdda66eda72: Status 404 returned error can't find the container with id 3400ea7efe246280879a61ccf340748f7cd8c2b9ed427ec93c5adcdda66eda72
Aug 04 03:45:55.101889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:55.101854    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:55.101963 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:55.101926    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:55.102003 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:55.101961    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:45:55.240335 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:55.240300    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40f269bf-5bc0-49cf-97ec-5bee4ebb5149" path="/var/lib/kubelet/pods/40f269bf-5bc0-49cf-97ec-5bee4ebb5149/volumes"
Aug 04 03:45:55.240842 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:55.240820    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac3746fa-27e3-4a26-858b-6bb35d846335" path="/var/lib/kubelet/pods/ac3746fa-27e3-4a26-858b-6bb35d846335/volumes"
Aug 04 03:45:55.757447 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:55.757395    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" event={"ID":"292573e8-6c48-4917-af9e-ec78c64d104c","Type":"ContainerStarted","Data":"6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c"}
Aug 04 03:45:55.757903 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:55.757478    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" event={"ID":"292573e8-6c48-4917-af9e-ec78c64d104c","Type":"ContainerStarted","Data":"3400ea7efe246280879a61ccf340748f7cd8c2b9ed427ec93c5adcdda66eda72"}
Aug 04 03:45:55.780969 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:55.780919    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" podStartSLOduration=1.780904472 podStartE2EDuration="1.780904472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:45:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:45:55.778791073 +0000 UTC m=+2173.076931586" watchObservedRunningTime="2026-08-04 03:45:55.780904472 +0000 UTC m=+2173.079044960"
Aug 04 03:45:55.944584 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:55.944524    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:56.949917 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:56.949889    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:57.118259 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:57.118219    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:57.217876 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:57.217781    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:57.767719 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:57.767683    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:57.768922 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:57.768901    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:45:57.817032 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:57.816993    2559 patch_prober.go:28] interesting pod/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 03:45:57.817189 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:57.817053    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 03:45:58.236435 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:58.236406    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:58.341991 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:58.341961    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:58.737707 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:58.737681    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:58.833422 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:58.833387    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:45:58.885892 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:58.885855    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"]
Aug 04 03:45:58.888736 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:58.888720    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:45:58.890976 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:58.890955    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-9e9f9650-dockercfg-4hcrq\""
Aug 04 03:45:58.891055 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:58.890982    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9e9f9650\""
Aug 04 03:45:58.898481 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:58.898459    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"]
Aug 04 03:45:59.004407 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.004324    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-plugins-config-volume\") pod \"payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg\" (UID: \"ab6dd093-3ce7-43ec-9717-1de54d92a7b1\") " pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:45:59.004407 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.004363    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2c97\" (UniqueName: \"kubernetes.io/projected/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-kube-api-access-x2c97\") pod \"payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg\" (UID: \"ab6dd093-3ce7-43ec-9717-1de54d92a7b1\") " pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:45:59.105246 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.105215    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-plugins-config-volume\") pod \"payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg\" (UID: \"ab6dd093-3ce7-43ec-9717-1de54d92a7b1\") " pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:45:59.105407 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.105264    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2c97\" (UniqueName: \"kubernetes.io/projected/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-kube-api-access-x2c97\") pod \"payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg\" (UID: \"ab6dd093-3ce7-43ec-9717-1de54d92a7b1\") " pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:45:59.106203 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.106159    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-plugins-config-volume\") pod \"payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg\" (UID: \"ab6dd093-3ce7-43ec-9717-1de54d92a7b1\") " pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:45:59.113346 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.113317    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2c97\" (UniqueName: \"kubernetes.io/projected/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-kube-api-access-x2c97\") pod \"payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg\" (UID: \"ab6dd093-3ce7-43ec-9717-1de54d92a7b1\") " pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:45:59.172910 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.172885    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:59.199676 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.199640    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:45:59.306924 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.306892    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-token\") pod \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") "
Aug 04 03:45:59.307332 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.306937    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-podinfo\") pod \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") "
Aug 04 03:45:59.307332 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.307067    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istiod-ca-cert\") pod \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") "
Aug 04 03:45:59.307332 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.307096    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-certs\") pod \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") "
Aug 04 03:45:59.307332 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.307122    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-credential-socket\") pod \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") "
Aug 04 03:45:59.307332 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.307150    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm9p5\" (UniqueName: \"kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-kube-api-access-hm9p5\") pod \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") "
Aug 04 03:45:59.307332 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.307193    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-data\") pod \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") "
Aug 04 03:45:59.307332 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.307227    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-socket\") pod \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") "
Aug 04 03:45:59.307332 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.307250    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-envoy\") pod \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\" (UID: \"7ad17e8b-8f62-42bb-b072-c098c284b6fa\") "
Aug 04 03:45:59.308386 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.308221    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ad17e8b-8f62-42bb-b072-c098c284b6fa" (UID: "7ad17e8b-8f62-42bb-b072-c098c284b6fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:45:59.308920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.308863    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ad17e8b-8f62-42bb-b072-c098c284b6fa" (UID: "7ad17e8b-8f62-42bb-b072-c098c284b6fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:45:59.309169 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.309130    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ad17e8b-8f62-42bb-b072-c098c284b6fa" (UID: "7ad17e8b-8f62-42bb-b072-c098c284b6fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:45:59.309236 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.309212    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ad17e8b-8f62-42bb-b072-c098c284b6fa" (UID: "7ad17e8b-8f62-42bb-b072-c098c284b6fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:45:59.309584 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.309269    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ad17e8b-8f62-42bb-b072-c098c284b6fa" (UID: "7ad17e8b-8f62-42bb-b072-c098c284b6fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:45:59.310189 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.310148    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ad17e8b-8f62-42bb-b072-c098c284b6fa" (UID: "7ad17e8b-8f62-42bb-b072-c098c284b6fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:45:59.310305 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.310224    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ad17e8b-8f62-42bb-b072-c098c284b6fa" (UID: "7ad17e8b-8f62-42bb-b072-c098c284b6fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:45:59.310619 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.310597    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-kube-api-access-hm9p5" (OuterVolumeSpecName: "kube-api-access-hm9p5") pod "7ad17e8b-8f62-42bb-b072-c098c284b6fa" (UID: "7ad17e8b-8f62-42bb-b072-c098c284b6fa"). InnerVolumeSpecName "kube-api-access-hm9p5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:45:59.311439 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.311414    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ad17e8b-8f62-42bb-b072-c098c284b6fa" (UID: "7ad17e8b-8f62-42bb-b072-c098c284b6fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 03:45:59.332054 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.332024    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"]
Aug 04 03:45:59.336770 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:45:59.336742    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6dd093_3ce7_43ec_9717_1de54d92a7b1.slice/crio-063f941e951db86ff89e0094ab5246591f56ccf7677317b07e775ccc811171b9 WatchSource:0}: Error finding container 063f941e951db86ff89e0094ab5246591f56ccf7677317b07e775ccc811171b9: Status 404 returned error can't find the container with id 063f941e951db86ff89e0094ab5246591f56ccf7677317b07e775ccc811171b9
Aug 04 03:45:59.408829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.408801    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:59.408829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.408826    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:59.408977 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.408834    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:59.408977 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.408845    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm9p5\" (UniqueName: \"kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-kube-api-access-hm9p5\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:59.408977 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.408854    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:59.408977 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.408862    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:59.408977 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.408870    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:59.408977 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.408878    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:59.408977 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.408887    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ad17e8b-8f62-42bb-b072-c098c284b6fa-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:45:59.785153 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.785114    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg" event={"ID":"ab6dd093-3ce7-43ec-9717-1de54d92a7b1","Type":"ContainerStarted","Data":"e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298"}
Aug 04 03:45:59.785358 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.785174    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:45:59.785358 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.785200    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg" event={"ID":"ab6dd093-3ce7-43ec-9717-1de54d92a7b1","Type":"ContainerStarted","Data":"063f941e951db86ff89e0094ab5246591f56ccf7677317b07e775ccc811171b9"}
Aug 04 03:45:59.786716 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.786690    2559 generic.go:356] "Generic (PLEG): container finished" podID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerID="659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176" exitCode=0
Aug 04 03:45:59.786793 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.786738    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"
Aug 04 03:45:59.786793 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.786775    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" event={"ID":"7ad17e8b-8f62-42bb-b072-c098c284b6fa","Type":"ContainerDied","Data":"659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176"}
Aug 04 03:45:59.786903 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.786831    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6" event={"ID":"7ad17e8b-8f62-42bb-b072-c098c284b6fa","Type":"ContainerDied","Data":"919507479b08d8da62afabddbe4e2f4a7502b13903e7e4b7a25d9b3cba27a47d"}
Aug 04 03:45:59.786903 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.786863    2559 scope.go:117] "RemoveContainer" containerID="659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176"
Aug 04 03:45:59.795700 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.795684    2559 scope.go:117] "RemoveContainer" containerID="74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f"
Aug 04 03:45:59.800871 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.800825    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg" podStartSLOduration=1.800808093 podStartE2EDuration="1.800808093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:45:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:45:59.800079147 +0000 UTC m=+2177.098219654" watchObservedRunningTime="2026-08-04 03:45:59.800808093 +0000 UTC m=+2177.098948582"
Aug 04 03:45:59.804328 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.804280    2559 scope.go:117] "RemoveContainer" containerID="659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176"
Aug 04 03:45:59.804610 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:45:59.804591    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176\": container with ID starting with 659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176 not found: ID does not exist" containerID="659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176"
Aug 04 03:45:59.804670 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.804619    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176"} err="failed to get container status \"659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176\": rpc error: code = NotFound desc = could not find container \"659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176\": container with ID starting with 659b8dcad50cd8ff8b89eeec9c27e710aa20fd77a74bed83d51ac846e53d9176 not found: ID does not exist"
Aug 04 03:45:59.804670 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.804635    2559 scope.go:117] "RemoveContainer" containerID="74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f"
Aug 04 03:45:59.804865 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:45:59.804848    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f\": container with ID starting with 74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f not found: ID does not exist" containerID="74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f"
Aug 04 03:45:59.804907 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.804871    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f"} err="failed to get container status \"74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f\": rpc error: code = NotFound desc = could not find container \"74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f\": container with ID starting with 74cdcab9682e27357d1bf9dae4dd2a6d4d5f17996effdaba3a07edd9e5dcc22f not found: ID does not exist"
Aug 04 03:45:59.813932 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.813909    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"]
Aug 04 03:45:59.817888 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:45:59.817864    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-822464dd-gw-openshift-default-79445455cb-nnmk6"]
Aug 04 03:46:01.239218 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:01.239184    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" path="/var/lib/kubelet/pods/7ad17e8b-8f62-42bb-b072-c098c284b6fa/volumes"
Aug 04 03:46:10.795508 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:10.795477    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:46:12.769782 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:12.769745    2559 patch_prober.go:28] interesting pod/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Aug 04 03:46:12.770208 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:12.769804    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Aug 04 03:46:12.857416 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:12.857385    2559 generic.go:356] "Generic (PLEG): container finished" podID="292573e8-6c48-4917-af9e-ec78c64d104c" containerID="6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c" exitCode=0
Aug 04 03:46:12.857614 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:12.857460    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" event={"ID":"292573e8-6c48-4917-af9e-ec78c64d104c","Type":"ContainerDied","Data":"6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c"}
Aug 04 03:46:12.857875 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:12.857861    2559 scope.go:117] "RemoveContainer" containerID="6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c"
Aug 04 03:46:12.858877 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:12.858843    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:46:12.858944 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:12.858911    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:46:12.858984 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:12.858943    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:46:13.864691 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:13.864657    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" event={"ID":"292573e8-6c48-4917-af9e-ec78c64d104c","Type":"ContainerStarted","Data":"43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55"}
Aug 04 03:46:13.945521 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:13.945479    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:46:13.945739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:13.945629    2559 patch_prober.go:28] interesting pod/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Aug 04 03:46:13.945739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:13.945675    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Aug 04 03:46:14.944780 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:14.944739    2559 patch_prober.go:28] interesting pod/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Aug 04 03:46:14.945285 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:14.944813    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Aug 04 03:46:15.945461 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:15.945404    2559 patch_prober.go:28] interesting pod/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Aug 04 03:46:15.945986 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:15.945483    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Aug 04 03:46:16.944869 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:16.944826    2559 patch_prober.go:28] interesting pod/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Aug 04 03:46:16.945146 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:16.945113    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Aug 04 03:46:17.948762 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:17.948727    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:46:17.949193 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:17.949019    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:46:17.949750 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:17.949733    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:46:22.811337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.811303    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"]
Aug 04 03:46:22.811738 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.811688    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy"
Aug 04 03:46:22.811738 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.811700    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy"
Aug 04 03:46:22.811738 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.811724    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy"
Aug 04 03:46:22.811738 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.811729    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy"
Aug 04 03:46:22.811875 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.811782    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy"
Aug 04 03:46:22.811875 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.811790    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad17e8b-8f62-42bb-b072-c098c284b6fa" containerName="istio-proxy"
Aug 04 03:46:22.814626 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.814609    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"
Aug 04 03:46:22.816990 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.816967    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 03:46:22.817156 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.817116    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7lrf4\""
Aug 04 03:46:22.821410 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.821387    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"]
Aug 04 03:46:22.925585 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:22.925529    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3fac1af-1000-4e71-8ab5-c90cf04aa1c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t\" (UID: \"d3fac1af-1000-4e71-8ab5-c90cf04aa1c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"
Aug 04 03:46:23.026190 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:23.026156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3fac1af-1000-4e71-8ab5-c90cf04aa1c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t\" (UID: \"d3fac1af-1000-4e71-8ab5-c90cf04aa1c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"
Aug 04 03:46:23.026833 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:23.026807    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3fac1af-1000-4e71-8ab5-c90cf04aa1c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t\" (UID: \"d3fac1af-1000-4e71-8ab5-c90cf04aa1c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"
Aug 04 03:46:23.125938 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:23.125854    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"
Aug 04 03:46:23.253185 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:23.253159    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"]
Aug 04 03:46:23.255309 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:46:23.255273    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3fac1af_1000_4e71_8ab5_c90cf04aa1c4.slice/crio-d4a03e7111c590cb9c695aafd77ccb3878fd2c668500fa922cbfde6b44dafedc WatchSource:0}: Error finding container d4a03e7111c590cb9c695aafd77ccb3878fd2c668500fa922cbfde6b44dafedc: Status 404 returned error can't find the container with id d4a03e7111c590cb9c695aafd77ccb3878fd2c668500fa922cbfde6b44dafedc
Aug 04 03:46:23.914490 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:23.914451    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t" event={"ID":"d3fac1af-1000-4e71-8ab5-c90cf04aa1c4","Type":"ContainerStarted","Data":"c1d510650cff14a366a7d687495bd97c6f14fe0381ac5b8311e5865b8ce06114"}
Aug 04 03:46:23.914893 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:23.914512    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t" event={"ID":"d3fac1af-1000-4e71-8ab5-c90cf04aa1c4","Type":"ContainerStarted","Data":"d4a03e7111c590cb9c695aafd77ccb3878fd2c668500fa922cbfde6b44dafedc"}
Aug 04 03:46:23.930270 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:23.930224    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t" podStartSLOduration=1.930205409 podStartE2EDuration="1.930205409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:46:23.929592933 +0000 UTC m=+2201.227733419" watchObservedRunningTime="2026-08-04 03:46:23.930205409 +0000 UTC m=+2201.228345899"
Aug 04 03:46:24.920588 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:24.920530    2559 generic.go:356] "Generic (PLEG): container finished" podID="d3fac1af-1000-4e71-8ab5-c90cf04aa1c4" containerID="c1d510650cff14a366a7d687495bd97c6f14fe0381ac5b8311e5865b8ce06114" exitCode=0
Aug 04 03:46:24.920981 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:24.920619    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t" event={"ID":"d3fac1af-1000-4e71-8ab5-c90cf04aa1c4","Type":"ContainerDied","Data":"c1d510650cff14a366a7d687495bd97c6f14fe0381ac5b8311e5865b8ce06114"}
Aug 04 03:46:26.058649 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:26.058625    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"
Aug 04 03:46:26.154596 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:26.154534    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3fac1af-1000-4e71-8ab5-c90cf04aa1c4-maas-api-service-ca\") pod \"d3fac1af-1000-4e71-8ab5-c90cf04aa1c4\" (UID: \"d3fac1af-1000-4e71-8ab5-c90cf04aa1c4\") "
Aug 04 03:46:26.154907 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:26.154883    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3fac1af-1000-4e71-8ab5-c90cf04aa1c4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d3fac1af-1000-4e71-8ab5-c90cf04aa1c4" (UID: "d3fac1af-1000-4e71-8ab5-c90cf04aa1c4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:46:26.256128 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:26.256102    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3fac1af-1000-4e71-8ab5-c90cf04aa1c4-maas-api-service-ca\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:26.931848 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:26.931795    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t" event={"ID":"d3fac1af-1000-4e71-8ab5-c90cf04aa1c4","Type":"ContainerDied","Data":"d4a03e7111c590cb9c695aafd77ccb3878fd2c668500fa922cbfde6b44dafedc"}
Aug 04 03:46:26.932024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:26.931866    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4a03e7111c590cb9c695aafd77ccb3878fd2c668500fa922cbfde6b44dafedc"
Aug 04 03:46:26.932024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:26.931821    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"
Aug 04 03:46:28.907126 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:28.907090    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:28.925323 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:28.925291    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"]
Aug 04 03:46:28.925777 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:28.925748    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg" podUID="ab6dd093-3ce7-43ec-9717-1de54d92a7b1" containerName="payload-processing" containerID="cri-o://e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298" gracePeriod=30
Aug 04 03:46:29.007356 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.007326    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:29.054686 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.054651    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"]
Aug 04 03:46:29.056445 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.056421    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9e9f9650-sbq5t"]
Aug 04 03:46:29.101147 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.101113    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:29.192769 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.192742    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:46:29.238867 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.238829    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3fac1af-1000-4e71-8ab5-c90cf04aa1c4" path="/var/lib/kubelet/pods/d3fac1af-1000-4e71-8ab5-c90cf04aa1c4/volumes"
Aug 04 03:46:29.285796 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.285759    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2c97\" (UniqueName: \"kubernetes.io/projected/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-kube-api-access-x2c97\") pod \"ab6dd093-3ce7-43ec-9717-1de54d92a7b1\" (UID: \"ab6dd093-3ce7-43ec-9717-1de54d92a7b1\") "
Aug 04 03:46:29.285945 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.285860    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-plugins-config-volume\") pod \"ab6dd093-3ce7-43ec-9717-1de54d92a7b1\" (UID: \"ab6dd093-3ce7-43ec-9717-1de54d92a7b1\") "
Aug 04 03:46:29.286208 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.286180    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab6dd093-3ce7-43ec-9717-1de54d92a7b1" (UID: "ab6dd093-3ce7-43ec-9717-1de54d92a7b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:46:29.287801 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.287780    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-kube-api-access-x2c97" (OuterVolumeSpecName: "kube-api-access-x2c97") pod "ab6dd093-3ce7-43ec-9717-1de54d92a7b1" (UID: "ab6dd093-3ce7-43ec-9717-1de54d92a7b1"). InnerVolumeSpecName "kube-api-access-x2c97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:46:29.387257 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.387217    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-plugins-config-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:29.387257 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.387252    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2c97\" (UniqueName: \"kubernetes.io/projected/ab6dd093-3ce7-43ec-9717-1de54d92a7b1-kube-api-access-x2c97\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:29.965355 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.965322    2559 generic.go:356] "Generic (PLEG): container finished" podID="ab6dd093-3ce7-43ec-9717-1de54d92a7b1" containerID="e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298" exitCode=0
Aug 04 03:46:29.965796 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.965366    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"
Aug 04 03:46:29.965796 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.965403    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg" event={"ID":"ab6dd093-3ce7-43ec-9717-1de54d92a7b1","Type":"ContainerDied","Data":"e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298"}
Aug 04 03:46:29.965796 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.965454    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg" event={"ID":"ab6dd093-3ce7-43ec-9717-1de54d92a7b1","Type":"ContainerDied","Data":"063f941e951db86ff89e0094ab5246591f56ccf7677317b07e775ccc811171b9"}
Aug 04 03:46:29.965796 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.965475    2559 scope.go:117] "RemoveContainer" containerID="e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298"
Aug 04 03:46:29.981047 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.981025    2559 scope.go:117] "RemoveContainer" containerID="e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298"
Aug 04 03:46:29.981333 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:46:29.981313    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298\": container with ID starting with e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298 not found: ID does not exist" containerID="e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298"
Aug 04 03:46:29.981400 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.981341    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298"} err="failed to get container status \"e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298\": rpc error: code = NotFound desc = could not find container \"e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298\": container with ID starting with e41d1d4ec50ad56af8c33bdfa5ed7c1918b21a2c42d0ff8e102ade6377960298 not found: ID does not exist"
Aug 04 03:46:29.992324 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.992289    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"]
Aug 04 03:46:29.993958 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:29.993937    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9e9f9650-8494998f74-sm7lg"]
Aug 04 03:46:30.131146 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.131113    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"]
Aug 04 03:46:30.131467 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.131438    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy" containerID="cri-o://43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55" gracePeriod=30
Aug 04 03:46:30.159422 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.159381    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:30.363585 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.363531    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:30.590745 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.590712    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:30.848472 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.848432    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"]
Aug 04 03:46:30.848980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.848954    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab6dd093-3ce7-43ec-9717-1de54d92a7b1" containerName="payload-processing"
Aug 04 03:46:30.848980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.848977    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab6dd093-3ce7-43ec-9717-1de54d92a7b1" containerName="payload-processing"
Aug 04 03:46:30.849157 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.849015    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3fac1af-1000-4e71-8ab5-c90cf04aa1c4" containerName="revoke-keys"
Aug 04 03:46:30.849157 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.849024    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3fac1af-1000-4e71-8ab5-c90cf04aa1c4" containerName="revoke-keys"
Aug 04 03:46:30.849157 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.849106    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab6dd093-3ce7-43ec-9717-1de54d92a7b1" containerName="payload-processing"
Aug 04 03:46:30.849157 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.849120    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3fac1af-1000-4e71-8ab5-c90cf04aa1c4" containerName="revoke-keys"
Aug 04 03:46:30.853937 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.853917    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:30.856232 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.856212    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-69e40c6b-openshift-default-dockercfg-hqrnf\""
Aug 04 03:46:30.864636 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.864613    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"]
Aug 04 03:46:30.875308 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.875272    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:30.899427 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.899402    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-data\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:30.899590 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.899446    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvlfm\" (UniqueName: \"kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-kube-api-access-pvlfm\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:30.899590 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.899511    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-credential-socket\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:30.899590 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.899580    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-podinfo\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:30.899716 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.899603    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220f8f51-7b40-4d98-967f-a92d9473a05a-istiod-ca-cert\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:30.899716 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.899623    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-envoy\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:30.899716 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.899683    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-socket\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:30.899826 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.899736    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-token\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:30.899826 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:30.899765    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-certs\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.000722 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.000683    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-token\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001196 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.000729    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-certs\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001196 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.000776    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-data\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001196 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.000831    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvlfm\" (UniqueName: \"kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-kube-api-access-pvlfm\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001196 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.000883    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-credential-socket\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001196 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.000917    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-podinfo\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001196 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.000940    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220f8f51-7b40-4d98-967f-a92d9473a05a-istiod-ca-cert\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001196 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.000969    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-envoy\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001196 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.001019    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-socket\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001684 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.001198    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-data\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001684 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.001265    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-certs\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001684 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.001321    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-credential-socket\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001684 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.001441    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-socket\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.001830 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.001701    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220f8f51-7b40-4d98-967f-a92d9473a05a-istiod-ca-cert\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.003305 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.003278    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-envoy\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.003497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.003478    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-podinfo\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.011484 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.011463    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-token\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.011616 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.011564    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvlfm\" (UniqueName: \"kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-kube-api-access-pvlfm\") pod \"e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") " pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.115405 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.115273    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:31.164037 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.163991    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:31.242689 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.242659    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab6dd093-3ce7-43ec-9717-1de54d92a7b1" path="/var/lib/kubelet/pods/ab6dd093-3ce7-43ec-9717-1de54d92a7b1/volumes"
Aug 04 03:46:31.309721 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.309592    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"]
Aug 04 03:46:31.312241 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:46:31.312214    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220f8f51_7b40_4d98_967f_a92d9473a05a.slice/crio-9b60414e010bb0ff56096bab769a66660831650accb9d57c2c0d5821dc0810f9 WatchSource:0}: Error finding container 9b60414e010bb0ff56096bab769a66660831650accb9d57c2c0d5821dc0810f9: Status 404 returned error can't find the container with id 9b60414e010bb0ff56096bab769a66660831650accb9d57c2c0d5821dc0810f9
Aug 04 03:46:31.314310 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.314282    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:46:31.314364 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.314349    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:46:31.314406 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.314397    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:46:31.978150 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.978100    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" event={"ID":"220f8f51-7b40-4d98-967f-a92d9473a05a","Type":"ContainerStarted","Data":"417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8"}
Aug 04 03:46:31.978321 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.978166    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" event={"ID":"220f8f51-7b40-4d98-967f-a92d9473a05a","Type":"ContainerStarted","Data":"9b60414e010bb0ff56096bab769a66660831650accb9d57c2c0d5821dc0810f9"}
Aug 04 03:46:31.998015 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:31.997957    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" podStartSLOduration=1.997941445 podStartE2EDuration="1.997941445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:46:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:46:31.996470699 +0000 UTC m=+2209.294611187" watchObservedRunningTime="2026-08-04 03:46:31.997941445 +0000 UTC m=+2209.296081933"
Aug 04 03:46:32.164629 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:32.164593    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:32.169347 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:32.169322    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:32.951832 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:32.951790    2559 patch_prober.go:28] interesting pod/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 03:46:32.952007 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:32.951865    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 03:46:32.983574 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:32.983525    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:32.984424 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:32.984406    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:33.351858 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:33.351820    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:33.451212 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:33.451182    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:34.476360 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:34.476314    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:34.698408 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:34.698374    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:34.968957 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:34.968920    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:35.065116 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.065077    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:46:35.394330 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.394302    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:46:35.451782 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.451749    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/292573e8-6c48-4917-af9e-ec78c64d104c-istio-podinfo\") pod \"292573e8-6c48-4917-af9e-ec78c64d104c\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") "
Aug 04 03:46:35.451952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.451785    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-data\") pod \"292573e8-6c48-4917-af9e-ec78c64d104c\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") "
Aug 04 03:46:35.451952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.451836    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-istio-token\") pod \"292573e8-6c48-4917-af9e-ec78c64d104c\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") "
Aug 04 03:46:35.451952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.451858    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/292573e8-6c48-4917-af9e-ec78c64d104c-istiod-ca-cert\") pod \"292573e8-6c48-4917-af9e-ec78c64d104c\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") "
Aug 04 03:46:35.451952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.451890    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-credential-socket\") pod \"292573e8-6c48-4917-af9e-ec78c64d104c\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") "
Aug 04 03:46:35.451952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.451914    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-socket\") pod \"292573e8-6c48-4917-af9e-ec78c64d104c\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") "
Aug 04 03:46:35.452225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.451978    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-certs\") pod \"292573e8-6c48-4917-af9e-ec78c64d104c\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") "
Aug 04 03:46:35.452225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.452001    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-envoy\") pod \"292573e8-6c48-4917-af9e-ec78c64d104c\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") "
Aug 04 03:46:35.452225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.452038    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr756\" (UniqueName: \"kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-kube-api-access-gr756\") pod \"292573e8-6c48-4917-af9e-ec78c64d104c\" (UID: \"292573e8-6c48-4917-af9e-ec78c64d104c\") "
Aug 04 03:46:35.452225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.452179    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "292573e8-6c48-4917-af9e-ec78c64d104c" (UID: "292573e8-6c48-4917-af9e-ec78c64d104c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:46:35.452225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.452210    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "292573e8-6c48-4917-af9e-ec78c64d104c" (UID: "292573e8-6c48-4917-af9e-ec78c64d104c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:46:35.452486 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.452294    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/292573e8-6c48-4917-af9e-ec78c64d104c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "292573e8-6c48-4917-af9e-ec78c64d104c" (UID: "292573e8-6c48-4917-af9e-ec78c64d104c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:46:35.452486 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.452410    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "292573e8-6c48-4917-af9e-ec78c64d104c" (UID: "292573e8-6c48-4917-af9e-ec78c64d104c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:46:35.452486 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.452461    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/292573e8-6c48-4917-af9e-ec78c64d104c-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:35.452486 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.452481    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:35.452749 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.452496    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:35.452823 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.452799    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-data" (OuterVolumeSpecName: "istio-data") pod "292573e8-6c48-4917-af9e-ec78c64d104c" (UID: "292573e8-6c48-4917-af9e-ec78c64d104c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:46:35.453996 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.453969    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/292573e8-6c48-4917-af9e-ec78c64d104c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "292573e8-6c48-4917-af9e-ec78c64d104c" (UID: "292573e8-6c48-4917-af9e-ec78c64d104c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 03:46:35.454114 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.454032    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-kube-api-access-gr756" (OuterVolumeSpecName: "kube-api-access-gr756") pod "292573e8-6c48-4917-af9e-ec78c64d104c" (UID: "292573e8-6c48-4917-af9e-ec78c64d104c"). InnerVolumeSpecName "kube-api-access-gr756". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:46:35.454114 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.454098    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "292573e8-6c48-4917-af9e-ec78c64d104c" (UID: "292573e8-6c48-4917-af9e-ec78c64d104c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:46:35.454194 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.454169    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-istio-token" (OuterVolumeSpecName: "istio-token") pod "292573e8-6c48-4917-af9e-ec78c64d104c" (UID: "292573e8-6c48-4917-af9e-ec78c64d104c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:46:35.553489 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.553412    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr756\" (UniqueName: \"kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-kube-api-access-gr756\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:35.553489 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.553443    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/292573e8-6c48-4917-af9e-ec78c64d104c-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:35.553489 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.553453    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:35.553489 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.553464    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/292573e8-6c48-4917-af9e-ec78c64d104c-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:35.553489 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.553471    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:35.553489 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.553479    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/292573e8-6c48-4917-af9e-ec78c64d104c-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:46:36.000025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:35.999988    2559 generic.go:356] "Generic (PLEG): container finished" podID="292573e8-6c48-4917-af9e-ec78c64d104c" containerID="43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55" exitCode=0
Aug 04 03:46:36.000210 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:36.000037    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"
Aug 04 03:46:36.000210 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:36.000055    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" event={"ID":"292573e8-6c48-4917-af9e-ec78c64d104c","Type":"ContainerDied","Data":"43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55"}
Aug 04 03:46:36.000210 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:36.000107    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv" event={"ID":"292573e8-6c48-4917-af9e-ec78c64d104c","Type":"ContainerDied","Data":"3400ea7efe246280879a61ccf340748f7cd8c2b9ed427ec93c5adcdda66eda72"}
Aug 04 03:46:36.000210 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:36.000139    2559 scope.go:117] "RemoveContainer" containerID="43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55"
Aug 04 03:46:36.009332 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:36.009308    2559 scope.go:117] "RemoveContainer" containerID="6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c"
Aug 04 03:46:36.016895 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:36.016875    2559 scope.go:117] "RemoveContainer" containerID="43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55"
Aug 04 03:46:36.017103 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:46:36.017086    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55\": container with ID starting with 43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55 not found: ID does not exist" containerID="43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55"
Aug 04 03:46:36.017143 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:36.017113    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55"} err="failed to get container status \"43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55\": rpc error: code = NotFound desc = could not find container \"43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55\": container with ID starting with 43ca358f1d61e8f14d78ef3e3f53464d04f44763c66d38b01eb8864880257d55 not found: ID does not exist"
Aug 04 03:46:36.017143 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:36.017135    2559 scope.go:117] "RemoveContainer" containerID="6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c"
Aug 04 03:46:36.017343 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:46:36.017329    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c\": container with ID starting with 6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c not found: ID does not exist" containerID="6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c"
Aug 04 03:46:36.017388 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:36.017346    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c"} err="failed to get container status \"6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c\": rpc error: code = NotFound desc = could not find container \"6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c\": container with ID starting with 6f61de1d1351359e1a0422bbefaf1c60ad6f4d8f758ba9114721bf0b9c49d60c not found: ID does not exist"
Aug 04 03:46:36.020601 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:36.020580    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"]
Aug 04 03:46:36.025748 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:36.025729    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9e9f9650-openshift-default-85dd4cc5fd-rngqv"]
Aug 04 03:46:37.239068 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:37.239037    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" path="/var/lib/kubelet/pods/292573e8-6c48-4917-af9e-ec78c64d104c/volumes"
Aug 04 03:46:47.984911 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:47.984870    2559 patch_prober.go:28] interesting pod/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Aug 04 03:46:47.985338 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:47.984944    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Aug 04 03:46:48.068515 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:48.068481    2559 generic.go:356] "Generic (PLEG): container finished" podID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerID="417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8" exitCode=0
Aug 04 03:46:48.068685 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:48.068558    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" event={"ID":"220f8f51-7b40-4d98-967f-a92d9473a05a","Type":"ContainerDied","Data":"417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8"}
Aug 04 03:46:48.068942 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:48.068928    2559 scope.go:117] "RemoveContainer" containerID="417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8"
Aug 04 03:46:48.070017 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:48.069988    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:46:48.070090 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:48.070062    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:46:48.070126 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:48.070094    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:46:49.074920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:49.074887    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" event={"ID":"220f8f51-7b40-4d98-967f-a92d9473a05a","Type":"ContainerStarted","Data":"badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851"}
Aug 04 03:46:49.164167 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:49.164129    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:49.164376 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:49.164350    2559 patch_prober.go:28] interesting pod/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Aug 04 03:46:49.164428 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:49.164402    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Aug 04 03:46:50.164733 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:50.164699    2559 patch_prober.go:28] interesting pod/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Aug 04 03:46:50.165133 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:50.164758    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Aug 04 03:46:51.165148 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:51.165115    2559 patch_prober.go:28] interesting pod/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Aug 04 03:46:51.165533 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:51.165173    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Aug 04 03:46:52.165028 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:52.164994    2559 patch_prober.go:28] interesting pod/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Aug 04 03:46:52.165200 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:52.165058    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Aug 04 03:46:53.168243 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:53.168203    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:53.168774 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:53.168645    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:53.169292 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:53.169273    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:46:58.206294 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.206260    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"]
Aug 04 03:46:58.206690 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.206650    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy"
Aug 04 03:46:58.206690 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.206665    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy"
Aug 04 03:46:58.206767 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.206737    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy"
Aug 04 03:46:58.209840 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.209823    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"
Aug 04 03:46:58.212410 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.212380    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7lrf4\""
Aug 04 03:46:58.212531 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.212477    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 03:46:58.217242 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.217217    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"]
Aug 04 03:46:58.273596 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.273556    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c638317d-4dcf-43fb-b08e-4791eb97b6b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj\" (UID: \"c638317d-4dcf-43fb-b08e-4791eb97b6b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"
Aug 04 03:46:58.374875 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.374841    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c638317d-4dcf-43fb-b08e-4791eb97b6b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj\" (UID: \"c638317d-4dcf-43fb-b08e-4791eb97b6b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"
Aug 04 03:46:58.375453 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.375434    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c638317d-4dcf-43fb-b08e-4791eb97b6b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj\" (UID: \"c638317d-4dcf-43fb-b08e-4791eb97b6b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"
Aug 04 03:46:58.521755 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.521724    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"
Aug 04 03:46:58.648651 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:58.648630    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"]
Aug 04 03:46:58.650612 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:46:58.650586    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc638317d_4dcf_43fb_b08e_4791eb97b6b2.slice/crio-fc0af8eada6ce5ce2e7d1e12a1b764677fe90710c21eda8492c63d35e830eb91 WatchSource:0}: Error finding container fc0af8eada6ce5ce2e7d1e12a1b764677fe90710c21eda8492c63d35e830eb91: Status 404 returned error can't find the container with id fc0af8eada6ce5ce2e7d1e12a1b764677fe90710c21eda8492c63d35e830eb91
Aug 04 03:46:59.125749 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:59.125667    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj" event={"ID":"c638317d-4dcf-43fb-b08e-4791eb97b6b2","Type":"ContainerStarted","Data":"0210d4fcc9aeb5069dad8cff954cd4bf81963f33e6a6fcb8df8c1fd4eb2d13e6"}
Aug 04 03:46:59.125749 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:59.125717    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj" event={"ID":"c638317d-4dcf-43fb-b08e-4791eb97b6b2","Type":"ContainerStarted","Data":"fc0af8eada6ce5ce2e7d1e12a1b764677fe90710c21eda8492c63d35e830eb91"}
Aug 04 03:46:59.141353 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:46:59.141302    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj" podStartSLOduration=1.14128616 podStartE2EDuration="1.14128616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:46:59.140901396 +0000 UTC m=+2236.439041906" watchObservedRunningTime="2026-08-04 03:46:59.14128616 +0000 UTC m=+2236.439426648"
Aug 04 03:47:00.131134 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:00.131095    2559 generic.go:356] "Generic (PLEG): container finished" podID="c638317d-4dcf-43fb-b08e-4791eb97b6b2" containerID="0210d4fcc9aeb5069dad8cff954cd4bf81963f33e6a6fcb8df8c1fd4eb2d13e6" exitCode=0
Aug 04 03:47:00.131531 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:00.131182    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj" event={"ID":"c638317d-4dcf-43fb-b08e-4791eb97b6b2","Type":"ContainerDied","Data":"0210d4fcc9aeb5069dad8cff954cd4bf81963f33e6a6fcb8df8c1fd4eb2d13e6"}
Aug 04 03:47:00.149711 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:00.149662    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:01.272574 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:01.272519    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"
Aug 04 03:47:01.300131 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:01.300100    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c638317d-4dcf-43fb-b08e-4791eb97b6b2-maas-api-service-ca\") pod \"c638317d-4dcf-43fb-b08e-4791eb97b6b2\" (UID: \"c638317d-4dcf-43fb-b08e-4791eb97b6b2\") "
Aug 04 03:47:01.300464 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:01.300441    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c638317d-4dcf-43fb-b08e-4791eb97b6b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c638317d-4dcf-43fb-b08e-4791eb97b6b2" (UID: "c638317d-4dcf-43fb-b08e-4791eb97b6b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:47:01.401086 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:01.401002    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c638317d-4dcf-43fb-b08e-4791eb97b6b2-maas-api-service-ca\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:47:02.141913 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:02.141879    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"
Aug 04 03:47:02.142079 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:02.141905    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj" event={"ID":"c638317d-4dcf-43fb-b08e-4791eb97b6b2","Type":"ContainerDied","Data":"fc0af8eada6ce5ce2e7d1e12a1b764677fe90710c21eda8492c63d35e830eb91"}
Aug 04 03:47:02.142079 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:02.141955    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc0af8eada6ce5ce2e7d1e12a1b764677fe90710c21eda8492c63d35e830eb91"
Aug 04 03:47:04.905244 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:04.905169    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:04.971711 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:04.971678    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:05.040480 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:05.040452    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:05.065525 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:05.065489    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"]
Aug 04 03:47:05.068574 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:05.068551    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-69e40c6b-5rfpj"]
Aug 04 03:47:05.139571 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:05.139526    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:05.197737 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:05.197656    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"]
Aug 04 03:47:05.197926 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:05.197901    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy" containerID="cri-o://badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851" gracePeriod=30
Aug 04 03:47:05.224929 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:05.224891    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:05.238555 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:05.238519    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c638317d-4dcf-43fb-b08e-4791eb97b6b2" path="/var/lib/kubelet/pods/c638317d-4dcf-43fb-b08e-4791eb97b6b2/volumes"
Aug 04 03:47:05.428955 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:05.428918    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:05.897991 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:05.897956    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:05.990744 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:05.990711    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:06.088286 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:06.088255    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:08.170696 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:08.170659    2559 patch_prober.go:28] interesting pod/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 03:47:08.171084 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:08.170716    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 03:47:10.455201 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.455177    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:47:10.478594 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.478505    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvlfm\" (UniqueName: \"kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-kube-api-access-pvlfm\") pod \"220f8f51-7b40-4d98-967f-a92d9473a05a\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") "
Aug 04 03:47:10.478731 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.478601    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-data\") pod \"220f8f51-7b40-4d98-967f-a92d9473a05a\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") "
Aug 04 03:47:10.478731 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.478626    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-credential-socket\") pod \"220f8f51-7b40-4d98-967f-a92d9473a05a\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") "
Aug 04 03:47:10.478731 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.478650    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220f8f51-7b40-4d98-967f-a92d9473a05a-istiod-ca-cert\") pod \"220f8f51-7b40-4d98-967f-a92d9473a05a\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") "
Aug 04 03:47:10.478731 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.478694    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-certs\") pod \"220f8f51-7b40-4d98-967f-a92d9473a05a\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") "
Aug 04 03:47:10.478950 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.478729    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-socket\") pod \"220f8f51-7b40-4d98-967f-a92d9473a05a\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") "
Aug 04 03:47:10.478950 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.478759    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-podinfo\") pod \"220f8f51-7b40-4d98-967f-a92d9473a05a\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") "
Aug 04 03:47:10.478950 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.478784    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-token\") pod \"220f8f51-7b40-4d98-967f-a92d9473a05a\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") "
Aug 04 03:47:10.478950 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.478873    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-envoy\") pod \"220f8f51-7b40-4d98-967f-a92d9473a05a\" (UID: \"220f8f51-7b40-4d98-967f-a92d9473a05a\") "
Aug 04 03:47:10.479142 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.478970    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "220f8f51-7b40-4d98-967f-a92d9473a05a" (UID: "220f8f51-7b40-4d98-967f-a92d9473a05a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:47:10.479142 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.479010    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "220f8f51-7b40-4d98-967f-a92d9473a05a" (UID: "220f8f51-7b40-4d98-967f-a92d9473a05a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:47:10.479142 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.479037    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/220f8f51-7b40-4d98-967f-a92d9473a05a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "220f8f51-7b40-4d98-967f-a92d9473a05a" (UID: "220f8f51-7b40-4d98-967f-a92d9473a05a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:47:10.479292 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.479276    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:47:10.479292 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.479284    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "220f8f51-7b40-4d98-967f-a92d9473a05a" (UID: "220f8f51-7b40-4d98-967f-a92d9473a05a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:47:10.480143 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.479301    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220f8f51-7b40-4d98-967f-a92d9473a05a-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:47:10.480143 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.479315    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:47:10.480143 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.479705    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-data" (OuterVolumeSpecName: "istio-data") pod "220f8f51-7b40-4d98-967f-a92d9473a05a" (UID: "220f8f51-7b40-4d98-967f-a92d9473a05a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:47:10.481408 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.481350    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-kube-api-access-pvlfm" (OuterVolumeSpecName: "kube-api-access-pvlfm") pod "220f8f51-7b40-4d98-967f-a92d9473a05a" (UID: "220f8f51-7b40-4d98-967f-a92d9473a05a"). InnerVolumeSpecName "kube-api-access-pvlfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:47:10.481715 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.481691    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-token" (OuterVolumeSpecName: "istio-token") pod "220f8f51-7b40-4d98-967f-a92d9473a05a" (UID: "220f8f51-7b40-4d98-967f-a92d9473a05a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:47:10.482353 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.482317    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "220f8f51-7b40-4d98-967f-a92d9473a05a" (UID: "220f8f51-7b40-4d98-967f-a92d9473a05a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:47:10.482471 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.482448    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "220f8f51-7b40-4d98-967f-a92d9473a05a" (UID: "220f8f51-7b40-4d98-967f-a92d9473a05a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 03:47:10.580494 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.580457    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:47:10.580494 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.580486    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:47:10.580494 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.580495    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:47:10.580739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.580506    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvlfm\" (UniqueName: \"kubernetes.io/projected/220f8f51-7b40-4d98-967f-a92d9473a05a-kube-api-access-pvlfm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:47:10.580739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.580516    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:47:10.580739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:10.580525    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220f8f51-7b40-4d98-967f-a92d9473a05a-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:47:11.187794 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.187759    2559 generic.go:356] "Generic (PLEG): container finished" podID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerID="badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851" exitCode=0
Aug 04 03:47:11.187987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.187807    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"
Aug 04 03:47:11.187987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.187847    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" event={"ID":"220f8f51-7b40-4d98-967f-a92d9473a05a","Type":"ContainerDied","Data":"badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851"}
Aug 04 03:47:11.187987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.187911    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj" event={"ID":"220f8f51-7b40-4d98-967f-a92d9473a05a","Type":"ContainerDied","Data":"9b60414e010bb0ff56096bab769a66660831650accb9d57c2c0d5821dc0810f9"}
Aug 04 03:47:11.187987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.187941    2559 scope.go:117] "RemoveContainer" containerID="badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851"
Aug 04 03:47:11.196807 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.196789    2559 scope.go:117] "RemoveContainer" containerID="417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8"
Aug 04 03:47:11.204617 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.204590    2559 scope.go:117] "RemoveContainer" containerID="badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851"
Aug 04 03:47:11.204866 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:47:11.204847    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851\": container with ID starting with badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851 not found: ID does not exist" containerID="badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851"
Aug 04 03:47:11.204939 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.204874    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851"} err="failed to get container status \"badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851\": rpc error: code = NotFound desc = could not find container \"badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851\": container with ID starting with badd3eb0c4e2ce22c473d448039547b7e93a31c03b41a5f435b9b377cdb65851 not found: ID does not exist"
Aug 04 03:47:11.204939 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.204890    2559 scope.go:117] "RemoveContainer" containerID="417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8"
Aug 04 03:47:11.205138 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:47:11.205121    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8\": container with ID starting with 417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8 not found: ID does not exist" containerID="417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8"
Aug 04 03:47:11.205180 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.205145    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8"} err="failed to get container status \"417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8\": rpc error: code = NotFound desc = could not find container \"417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8\": container with ID starting with 417caac78b9d90e59c8cf972bf6d6e2e6010e26a85254006b9b0dc1d03c360f8 not found: ID does not exist"
Aug 04 03:47:11.208764 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.208743    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"]
Aug 04 03:47:11.218174 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.218144    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-69e40c6b-openshift-default-6954bdf784-jc9sj"]
Aug 04 03:47:11.238926 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:11.238900    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" path="/var/lib/kubelet/pods/220f8f51-7b40-4d98-967f-a92d9473a05a/volumes"
Aug 04 03:47:27.888279 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:27.888242    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:33.780225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:33.780186    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:33.885099 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:33.885064    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:33.977018 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:33.976989    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:34.077197 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:34.077107    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:47:34.471914 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:47:34.471836    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:00.384292 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:00.384260    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:06.478440 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:06.478405    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:06.593679 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:06.593646    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:06.678404 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:06.678367    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:22.567323 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:22.567285    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:28.555954 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:28.555918    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:28.619400 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:28.619345    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:28.707198 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:28.707165    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:28.807017 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:28.806943    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:39.259668 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:39.259567    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:50.259921 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:50.259887    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:56.152947 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:56.152911    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:56.273846 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:56.273814    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:56.352718 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:56.352675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:56.493188 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:56.493158    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:57.033254 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:57.033223    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:57.124450 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:57.124414    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:57.225334 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:57.225303    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:48:58.639922 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:48:58.639891    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:04.150720 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:04.150683    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:05.433603 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:05.433557    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:11.155223 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.155193    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:11.213237 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.213204    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:11.422057 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.421968    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:11.657225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.657184    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"]
Aug 04 03:49:11.657530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.657518    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy"
Aug 04 03:49:11.657530 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.657530    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy"
Aug 04 03:49:11.657682 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.657563    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy"
Aug 04 03:49:11.657682 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.657572    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy"
Aug 04 03:49:11.657682 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.657579    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c638317d-4dcf-43fb-b08e-4791eb97b6b2" containerName="revoke-keys"
Aug 04 03:49:11.657682 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.657588    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c638317d-4dcf-43fb-b08e-4791eb97b6b2" containerName="revoke-keys"
Aug 04 03:49:11.657682 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.657642    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c638317d-4dcf-43fb-b08e-4791eb97b6b2" containerName="revoke-keys"
Aug 04 03:49:11.657682 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.657650    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy"
Aug 04 03:49:11.657682 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.657659    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="292573e8-6c48-4917-af9e-ec78c64d104c" containerName="istio-proxy"
Aug 04 03:49:11.660606 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.660569    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.663337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.663222    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-bb6bbf1b-openshift-default-dockercfg-9p7zd\""
Aug 04 03:49:11.681718 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.681657    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"]
Aug 04 03:49:11.686511 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.686486    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:11.745305 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.745267    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-credential-socket\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.745497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.745318    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-data\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.745497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.745366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgx6p\" (UniqueName: \"kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-kube-api-access-qgx6p\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.745497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.745390    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-certs\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.745497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.745463    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-envoy\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.745497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.745488    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-token\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.745773 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.745519    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-podinfo\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.745773 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.745581    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a948a631-7e10-47c8-a00a-47c487dd1dc1-istiod-ca-cert\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.745773 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.745669    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-socket\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.844895 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.844856    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:11.846950 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.846922    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a948a631-7e10-47c8-a00a-47c487dd1dc1-istiod-ca-cert\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847106 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.846997    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-socket\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847106 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-credential-socket\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847106 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847064    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-data\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847106 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847094    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgx6p\" (UniqueName: \"kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-kube-api-access-qgx6p\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847316 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-certs\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847316 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847170    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-envoy\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847316 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847196    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-token\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847316 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847230    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-podinfo\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847566 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847456    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-credential-socket\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847566 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847481    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-socket\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847691 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847622    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-certs\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847723    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a948a631-7e10-47c8-a00a-47c487dd1dc1-istiod-ca-cert\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.847848 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.847828    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-data\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.849457 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.849434    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-envoy\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.849665 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.849648    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-podinfo\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.857776 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.857752    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgx6p\" (UniqueName: \"kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-kube-api-access-qgx6p\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.858095 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.858078    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-token\") pod \"e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") " pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:11.975027 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:11.974937    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:12.120275 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:12.120250    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"]
Aug 04 03:49:12.121855 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:49:12.121829    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda948a631_7e10_47c8_a00a_47c487dd1dc1.slice/crio-8e1d91ce709c6546f85b5554a5f795c870f4f85924b3b070fb5798b8c5a490b7 WatchSource:0}: Error finding container 8e1d91ce709c6546f85b5554a5f795c870f4f85924b3b070fb5798b8c5a490b7: Status 404 returned error can't find the container with id 8e1d91ce709c6546f85b5554a5f795c870f4f85924b3b070fb5798b8c5a490b7
Aug 04 03:49:12.123932 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:12.123895    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:49:12.124047 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:12.123975    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:49:12.124047 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:12.124019    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:49:12.799526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:12.799488    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" event={"ID":"a948a631-7e10-47c8-a00a-47c487dd1dc1","Type":"ContainerStarted","Data":"25dae57ac85c995e80b76b0c0a1f1f050a3c0138455808e438a594ba97effc35"}
Aug 04 03:49:12.799916 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:12.799556    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" event={"ID":"a948a631-7e10-47c8-a00a-47c487dd1dc1","Type":"ContainerStarted","Data":"8e1d91ce709c6546f85b5554a5f795c870f4f85924b3b070fb5798b8c5a490b7"}
Aug 04 03:49:12.819433 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:12.819384    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" podStartSLOduration=1.819350263 podStartE2EDuration="1.819350263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:49:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:49:12.817685977 +0000 UTC m=+2370.115826485" watchObservedRunningTime="2026-08-04 03:49:12.819350263 +0000 UTC m=+2370.117490746"
Aug 04 03:49:12.976045 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:12.976001    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:13.980293 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:13.980263    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:14.714199 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:14.714154    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:14.813831 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:14.813802    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:14.815165 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:14.815140    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:14.815660 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:14.815644    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:15.832712 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:15.832681    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:16.265043 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.265009    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:16.419002 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.418941    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:16.514663 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.514628    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:16.519764 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.519702    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"]
Aug 04 03:49:16.520074 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.520061    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy"
Aug 04 03:49:16.520126 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.520075    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy"
Aug 04 03:49:16.520168 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.520148    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="220f8f51-7b40-4d98-967f-a92d9473a05a" containerName="istio-proxy"
Aug 04 03:49:16.523275 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.523259    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:16.526374 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.526354    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-bb6bbf1b-dockercfg-9dmz6\""
Aug 04 03:49:16.526374 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.526366    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-bb6bbf1b\""
Aug 04 03:49:16.537735 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.537712    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"]
Aug 04 03:49:16.553529 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.553492    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"]
Aug 04 03:49:16.557054 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.557032    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:16.567748 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.567727    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"]
Aug 04 03:49:16.692487 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.692452    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnrkk\" (UniqueName: \"kubernetes.io/projected/fc52894c-e96c-4fcc-adb6-73031a191cdd-kube-api-access-fnrkk\") pod \"payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6\" (UID: \"fc52894c-e96c-4fcc-adb6-73031a191cdd\") " pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:16.692721 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.692500    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb\" (UID: \"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:16.692721 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.692535    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc52894c-e96c-4fcc-adb6-73031a191cdd-plugins-config-volume\") pod \"payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6\" (UID: \"fc52894c-e96c-4fcc-adb6-73031a191cdd\") " pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:16.692721 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.692603    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjpx4\" (UniqueName: \"kubernetes.io/projected/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-kube-api-access-kjpx4\") pod \"payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb\" (UID: \"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:16.793777 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.793690    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnrkk\" (UniqueName: \"kubernetes.io/projected/fc52894c-e96c-4fcc-adb6-73031a191cdd-kube-api-access-fnrkk\") pod \"payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6\" (UID: \"fc52894c-e96c-4fcc-adb6-73031a191cdd\") " pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:16.793777 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.793735    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb\" (UID: \"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:16.793777 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.793756    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc52894c-e96c-4fcc-adb6-73031a191cdd-plugins-config-volume\") pod \"payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6\" (UID: \"fc52894c-e96c-4fcc-adb6-73031a191cdd\") " pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:16.793777 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.793771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjpx4\" (UniqueName: \"kubernetes.io/projected/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-kube-api-access-kjpx4\") pod \"payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb\" (UID: \"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:16.794535 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.794509    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb\" (UID: \"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:16.794713 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.794689    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc52894c-e96c-4fcc-adb6-73031a191cdd-plugins-config-volume\") pod \"payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6\" (UID: \"fc52894c-e96c-4fcc-adb6-73031a191cdd\") " pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:16.801961 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.801933    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnrkk\" (UniqueName: \"kubernetes.io/projected/fc52894c-e96c-4fcc-adb6-73031a191cdd-kube-api-access-fnrkk\") pod \"payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6\" (UID: \"fc52894c-e96c-4fcc-adb6-73031a191cdd\") " pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:16.802066 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.801993    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjpx4\" (UniqueName: \"kubernetes.io/projected/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-kube-api-access-kjpx4\") pod \"payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb\" (UID: \"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:16.835463 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.835432    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:16.867626 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.867595    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:16.986978 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:16.986949    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"]
Aug 04 03:49:16.988794 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:49:16.988762    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecbf1013_d3e9_48b0_ab8d_39b900dd6ac5.slice/crio-66bf27408ac25a08f69aa5bba2bfbe509f59ef1f3cb742e41ad7b15e5e61ff25 WatchSource:0}: Error finding container 66bf27408ac25a08f69aa5bba2bfbe509f59ef1f3cb742e41ad7b15e5e61ff25: Status 404 returned error can't find the container with id 66bf27408ac25a08f69aa5bba2bfbe509f59ef1f3cb742e41ad7b15e5e61ff25
Aug 04 03:49:17.031793 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:17.031694    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"]
Aug 04 03:49:17.034678 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:49:17.034645    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc52894c_e96c_4fcc_adb6_73031a191cdd.slice/crio-11cc4b8380a1e23ec950fb130fd0d6df035e45b1daa56557cdcf716ac59737c5 WatchSource:0}: Error finding container 11cc4b8380a1e23ec950fb130fd0d6df035e45b1daa56557cdcf716ac59737c5: Status 404 returned error can't find the container with id 11cc4b8380a1e23ec950fb130fd0d6df035e45b1daa56557cdcf716ac59737c5
Aug 04 03:49:17.829773 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:17.829738    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb" event={"ID":"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5","Type":"ContainerStarted","Data":"7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3"}
Aug 04 03:49:17.830039 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:17.830023    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:17.830178 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:17.830142    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb" event={"ID":"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5","Type":"ContainerStarted","Data":"66bf27408ac25a08f69aa5bba2bfbe509f59ef1f3cb742e41ad7b15e5e61ff25"}
Aug 04 03:49:17.831577 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:17.831534    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6" event={"ID":"fc52894c-e96c-4fcc-adb6-73031a191cdd","Type":"ContainerStarted","Data":"7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153"}
Aug 04 03:49:17.831745 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:17.831724    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6" event={"ID":"fc52894c-e96c-4fcc-adb6-73031a191cdd","Type":"ContainerStarted","Data":"11cc4b8380a1e23ec950fb130fd0d6df035e45b1daa56557cdcf716ac59737c5"}
Aug 04 03:49:17.832094 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:17.831760    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:17.843794 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:17.843753    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb" podStartSLOduration=1.843736166 podStartE2EDuration="1.843736166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:49:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:49:17.84329227 +0000 UTC m=+2375.141432759" watchObservedRunningTime="2026-08-04 03:49:17.843736166 +0000 UTC m=+2375.141876654"
Aug 04 03:49:17.861073 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:17.861030    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6" podStartSLOduration=1.861015273 podStartE2EDuration="1.861015273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:49:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:49:17.858934659 +0000 UTC m=+2375.157075145" watchObservedRunningTime="2026-08-04 03:49:17.861015273 +0000 UTC m=+2375.159155760"
Aug 04 03:49:27.891201 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:27.891164    2559 generic.go:356] "Generic (PLEG): container finished" podID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerID="25dae57ac85c995e80b76b0c0a1f1f050a3c0138455808e438a594ba97effc35" exitCode=0
Aug 04 03:49:27.891715 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:27.891217    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" event={"ID":"a948a631-7e10-47c8-a00a-47c487dd1dc1","Type":"ContainerDied","Data":"25dae57ac85c995e80b76b0c0a1f1f050a3c0138455808e438a594ba97effc35"}
Aug 04 03:49:27.891760 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:27.891730    2559 scope.go:117] "RemoveContainer" containerID="25dae57ac85c995e80b76b0c0a1f1f050a3c0138455808e438a594ba97effc35"
Aug 04 03:49:27.892743 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:27.892712    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:49:27.892828 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:27.892780    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:49:27.892828 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:27.892811    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:49:28.840806 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:28.840770    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:28.841733 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:28.841711    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:28.897948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:28.897919    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" event={"ID":"a948a631-7e10-47c8-a00a-47c487dd1dc1","Type":"ContainerStarted","Data":"c1384cbcff2c3b76dab15ebee1090c25127101b1a0244e445a337573b34c5428"}
Aug 04 03:49:28.975939 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:28.975910    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:28.977203 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:28.977168    2559 patch_prober.go:28] interesting pod/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Aug 04 03:49:28.977433 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:28.977401    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Aug 04 03:49:29.814069 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:29.814023    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:29.976217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:29.976188    2559 patch_prober.go:28] interesting pod/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Aug 04 03:49:29.976669 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:29.976641    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Aug 04 03:49:30.975737 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:30.975700    2559 patch_prober.go:28] interesting pod/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Aug 04 03:49:30.975910 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:30.975758    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Aug 04 03:49:31.975384 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:31.975343    2559 patch_prober.go:28] interesting pod/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Aug 04 03:49:31.975822 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:31.975401    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Aug 04 03:49:32.979513 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:32.979479    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:32.980553 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:32.980525    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:49:39.661457 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:39.661424    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:40.313262 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:40.313230    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:40.408390 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:40.408353    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:40.716719 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:40.716620    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:46.572808 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:46.572777    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:46.823957 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:46.823870    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:57.413238 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.413206    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:57.432392 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.432365    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"]
Aug 04 03:49:57.432763 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.432733    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6" podUID="fc52894c-e96c-4fcc-adb6-73031a191cdd" containerName="payload-processing" containerID="cri-o://7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153" gracePeriod=30
Aug 04 03:49:57.441959 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.441933    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"]
Aug 04 03:49:57.442180 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.442151    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb" podUID="ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5" containerName="payload-pre-processing" containerID="cri-o://7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3" gracePeriod=30
Aug 04 03:49:57.478749 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.478712    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:57.569012 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.568975    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:57.633645 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.633573    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:57.696324 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.696298    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:57.726391 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.726370    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:57.879035 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.879008    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-plugins-config-volume\") pod \"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5\" (UID: \"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5\") "
Aug 04 03:49:57.879189 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.879067    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnrkk\" (UniqueName: \"kubernetes.io/projected/fc52894c-e96c-4fcc-adb6-73031a191cdd-kube-api-access-fnrkk\") pod \"fc52894c-e96c-4fcc-adb6-73031a191cdd\" (UID: \"fc52894c-e96c-4fcc-adb6-73031a191cdd\") "
Aug 04 03:49:57.879189 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.879094    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc52894c-e96c-4fcc-adb6-73031a191cdd-plugins-config-volume\") pod \"fc52894c-e96c-4fcc-adb6-73031a191cdd\" (UID: \"fc52894c-e96c-4fcc-adb6-73031a191cdd\") "
Aug 04 03:49:57.879189 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.879129    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjpx4\" (UniqueName: \"kubernetes.io/projected/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-kube-api-access-kjpx4\") pod \"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5\" (UID: \"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5\") "
Aug 04 03:49:57.879449 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.879414    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5" (UID: "ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:49:57.879595 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.879456    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc52894c-e96c-4fcc-adb6-73031a191cdd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc52894c-e96c-4fcc-adb6-73031a191cdd" (UID: "fc52894c-e96c-4fcc-adb6-73031a191cdd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:49:57.881210 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.881190    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc52894c-e96c-4fcc-adb6-73031a191cdd-kube-api-access-fnrkk" (OuterVolumeSpecName: "kube-api-access-fnrkk") pod "fc52894c-e96c-4fcc-adb6-73031a191cdd" (UID: "fc52894c-e96c-4fcc-adb6-73031a191cdd"). InnerVolumeSpecName "kube-api-access-fnrkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:49:57.881279 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.881206    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-kube-api-access-kjpx4" (OuterVolumeSpecName: "kube-api-access-kjpx4") pod "ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5" (UID: "ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5"). InnerVolumeSpecName "kube-api-access-kjpx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:49:57.944349 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.944289    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:49:57.979884 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.979861    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnrkk\" (UniqueName: \"kubernetes.io/projected/fc52894c-e96c-4fcc-adb6-73031a191cdd-kube-api-access-fnrkk\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:49:57.979884 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.979884    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc52894c-e96c-4fcc-adb6-73031a191cdd-plugins-config-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:49:57.980040 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.979894    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjpx4\" (UniqueName: \"kubernetes.io/projected/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-kube-api-access-kjpx4\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:49:57.980040 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:57.979903    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5-plugins-config-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:49:58.041703 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.041671    2559 generic.go:356] "Generic (PLEG): container finished" podID="fc52894c-e96c-4fcc-adb6-73031a191cdd" containerID="7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153" exitCode=0
Aug 04 03:49:58.041859 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.041704    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6" event={"ID":"fc52894c-e96c-4fcc-adb6-73031a191cdd","Type":"ContainerDied","Data":"7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153"}
Aug 04 03:49:58.041859 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.041725    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"
Aug 04 03:49:58.041859 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.041771    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6" event={"ID":"fc52894c-e96c-4fcc-adb6-73031a191cdd","Type":"ContainerDied","Data":"11cc4b8380a1e23ec950fb130fd0d6df035e45b1daa56557cdcf716ac59737c5"}
Aug 04 03:49:58.041859 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.041803    2559 scope.go:117] "RemoveContainer" containerID="7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153"
Aug 04 03:49:58.043121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.043103    2559 generic.go:356] "Generic (PLEG): container finished" podID="ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5" containerID="7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3" exitCode=0
Aug 04 03:49:58.043208 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.043124    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb" event={"ID":"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5","Type":"ContainerDied","Data":"7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3"}
Aug 04 03:49:58.043208 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.043148    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"
Aug 04 03:49:58.043208 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.043173    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb" event={"ID":"ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5","Type":"ContainerDied","Data":"66bf27408ac25a08f69aa5bba2bfbe509f59ef1f3cb742e41ad7b15e5e61ff25"}
Aug 04 03:49:58.053301 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.053275    2559 scope.go:117] "RemoveContainer" containerID="7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153"
Aug 04 03:49:58.053640 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:49:58.053597    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153\": container with ID starting with 7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153 not found: ID does not exist" containerID="7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153"
Aug 04 03:49:58.053740 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.053637    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153"} err="failed to get container status \"7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153\": rpc error: code = NotFound desc = could not find container \"7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153\": container with ID starting with 7f7f0dd24e9b86b96cf3f52743b024e904557b040b192088d27cd8c974d77153 not found: ID does not exist"
Aug 04 03:49:58.053740 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.053660    2559 scope.go:117] "RemoveContainer" containerID="7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3"
Aug 04 03:49:58.061987 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.061972    2559 scope.go:117] "RemoveContainer" containerID="7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3"
Aug 04 03:49:58.062265 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:49:58.062246    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3\": container with ID starting with 7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3 not found: ID does not exist" containerID="7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3"
Aug 04 03:49:58.062310 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.062273    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3"} err="failed to get container status \"7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3\": rpc error: code = NotFound desc = could not find container \"7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3\": container with ID starting with 7f01751f3b6b37712c2b5e2c70685d6d89af7ff2d0b26d72706153cf6edaa5d3 not found: ID does not exist"
Aug 04 03:49:58.080367 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.080340    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"]
Aug 04 03:49:58.087179 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.087159    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bb6bbf1b-5479f5d678-pcvz6"]
Aug 04 03:49:58.109838 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.109814    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"]
Aug 04 03:49:58.126467 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:58.126438    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bb6bbf1b-7444944b48-gh5hb"]
Aug 04 03:49:59.238112 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:59.238070    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5" path="/var/lib/kubelet/pods/ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5/volumes"
Aug 04 03:49:59.238610 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:49:59.238580    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc52894c-e96c-4fcc-adb6-73031a191cdd" path="/var/lib/kubelet/pods/fc52894c-e96c-4fcc-adb6-73031a191cdd/volumes"
Aug 04 03:50:03.675921 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:03.675886    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:03.727663 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:03.727621    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:03.902509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:03.902476    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"]
Aug 04 03:50:03.902788 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:03.902760    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerName="istio-proxy" containerID="cri-o://c1384cbcff2c3b76dab15ebee1090c25127101b1a0244e445a337573b34c5428" gracePeriod=30
Aug 04 03:50:03.925573 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:03.925519    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:04.145295 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:04.145258    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:04.270601 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:04.270558    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:04.369772 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:04.369742    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:04.471966 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:04.471873    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:09.103050 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.103019    2559 generic.go:356] "Generic (PLEG): container finished" podID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerID="c1384cbcff2c3b76dab15ebee1090c25127101b1a0244e445a337573b34c5428" exitCode=0
Aug 04 03:50:09.103464 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.103088    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" event={"ID":"a948a631-7e10-47c8-a00a-47c487dd1dc1","Type":"ContainerDied","Data":"c1384cbcff2c3b76dab15ebee1090c25127101b1a0244e445a337573b34c5428"}
Aug 04 03:50:09.103464 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.103146    2559 scope.go:117] "RemoveContainer" containerID="25dae57ac85c995e80b76b0c0a1f1f050a3c0138455808e438a594ba97effc35"
Aug 04 03:50:09.150628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.150602    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:50:09.175231 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.175178    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-podinfo\") pod \"a948a631-7e10-47c8-a00a-47c487dd1dc1\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") "
Aug 04 03:50:09.175231 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.175212    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgx6p\" (UniqueName: \"kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-kube-api-access-qgx6p\") pod \"a948a631-7e10-47c8-a00a-47c487dd1dc1\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") "
Aug 04 03:50:09.175434 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.175238    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-envoy\") pod \"a948a631-7e10-47c8-a00a-47c487dd1dc1\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") "
Aug 04 03:50:09.175434 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.175282    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a948a631-7e10-47c8-a00a-47c487dd1dc1-istiod-ca-cert\") pod \"a948a631-7e10-47c8-a00a-47c487dd1dc1\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") "
Aug 04 03:50:09.175434 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.175315    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-credential-socket\") pod \"a948a631-7e10-47c8-a00a-47c487dd1dc1\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") "
Aug 04 03:50:09.175434 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.175342    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-certs\") pod \"a948a631-7e10-47c8-a00a-47c487dd1dc1\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") "
Aug 04 03:50:09.175434 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.175402    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-token\") pod \"a948a631-7e10-47c8-a00a-47c487dd1dc1\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") "
Aug 04 03:50:09.175434 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.175426    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-socket\") pod \"a948a631-7e10-47c8-a00a-47c487dd1dc1\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") "
Aug 04 03:50:09.175842 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.175505    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-data\") pod \"a948a631-7e10-47c8-a00a-47c487dd1dc1\" (UID: \"a948a631-7e10-47c8-a00a-47c487dd1dc1\") "
Aug 04 03:50:09.176008 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.175929    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a948a631-7e10-47c8-a00a-47c487dd1dc1" (UID: "a948a631-7e10-47c8-a00a-47c487dd1dc1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:50:09.176160 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.176137    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a948a631-7e10-47c8-a00a-47c487dd1dc1" (UID: "a948a631-7e10-47c8-a00a-47c487dd1dc1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:50:09.176337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.176316    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a948a631-7e10-47c8-a00a-47c487dd1dc1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a948a631-7e10-47c8-a00a-47c487dd1dc1" (UID: "a948a631-7e10-47c8-a00a-47c487dd1dc1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:50:09.176404 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.176370    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a948a631-7e10-47c8-a00a-47c487dd1dc1" (UID: "a948a631-7e10-47c8-a00a-47c487dd1dc1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:50:09.176637 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.176614    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-data" (OuterVolumeSpecName: "istio-data") pod "a948a631-7e10-47c8-a00a-47c487dd1dc1" (UID: "a948a631-7e10-47c8-a00a-47c487dd1dc1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:50:09.177567 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.177517    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-kube-api-access-qgx6p" (OuterVolumeSpecName: "kube-api-access-qgx6p") pod "a948a631-7e10-47c8-a00a-47c487dd1dc1" (UID: "a948a631-7e10-47c8-a00a-47c487dd1dc1"). InnerVolumeSpecName "kube-api-access-qgx6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:50:09.177666 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.177561    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a948a631-7e10-47c8-a00a-47c487dd1dc1" (UID: "a948a631-7e10-47c8-a00a-47c487dd1dc1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 03:50:09.177666 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.177631    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-token" (OuterVolumeSpecName: "istio-token") pod "a948a631-7e10-47c8-a00a-47c487dd1dc1" (UID: "a948a631-7e10-47c8-a00a-47c487dd1dc1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:50:09.177839 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.177821    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a948a631-7e10-47c8-a00a-47c487dd1dc1" (UID: "a948a631-7e10-47c8-a00a-47c487dd1dc1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:50:09.276829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.276796    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:50:09.276829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.276823    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:50:09.276829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.276833    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgx6p\" (UniqueName: \"kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-kube-api-access-qgx6p\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:50:09.277034 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.276842    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:50:09.277034 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.276852    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a948a631-7e10-47c8-a00a-47c487dd1dc1-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:50:09.277034 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.276860    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:50:09.277034 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.276869    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:50:09.277034 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.276877    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a948a631-7e10-47c8-a00a-47c487dd1dc1-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:50:09.277034 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:09.276884    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a948a631-7e10-47c8-a00a-47c487dd1dc1-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:50:10.110490 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:10.110456    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"
Aug 04 03:50:10.110490 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:10.110467    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm" event={"ID":"a948a631-7e10-47c8-a00a-47c487dd1dc1","Type":"ContainerDied","Data":"8e1d91ce709c6546f85b5554a5f795c870f4f85924b3b070fb5798b8c5a490b7"}
Aug 04 03:50:10.110975 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:10.110589    2559 scope.go:117] "RemoveContainer" containerID="c1384cbcff2c3b76dab15ebee1090c25127101b1a0244e445a337573b34c5428"
Aug 04 03:50:10.120029 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:10.120004    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:10.126697 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:10.126673    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"]
Aug 04 03:50:10.130633 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:10.130612    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bb6bbf1b-openshift-default-65c4bb6484-7b6fm"]
Aug 04 03:50:10.179479 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:10.179433    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:10.262995 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:10.262933    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:10.362897 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:10.362792    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:10.463976 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:10.463924    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:11.238557 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:11.238508    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" path="/var/lib/kubelet/pods/a948a631-7e10-47c8-a00a-47c487dd1dc1/volumes"
Aug 04 03:50:12.377244 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:12.377194    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:15.988760 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:15.988727    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:16.088616 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:16.088571    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:50:44.170109 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:44.170075    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"]
Aug 04 03:50:44.170632 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:44.170302    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v" podUID="e559706b-fe79-46cf-9998-b0a331522a8c" containerName="manager" containerID="cri-o://aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5" gracePeriod=10
Aug 04 03:50:44.920346 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:44.920321    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:50:45.093363 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.093287    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wvn5\" (UniqueName: \"kubernetes.io/projected/e559706b-fe79-46cf-9998-b0a331522a8c-kube-api-access-8wvn5\") pod \"e559706b-fe79-46cf-9998-b0a331522a8c\" (UID: \"e559706b-fe79-46cf-9998-b0a331522a8c\") "
Aug 04 03:50:45.093509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.093378    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e559706b-fe79-46cf-9998-b0a331522a8c-extensions-socket-volume\") pod \"e559706b-fe79-46cf-9998-b0a331522a8c\" (UID: \"e559706b-fe79-46cf-9998-b0a331522a8c\") "
Aug 04 03:50:45.093758 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.093725    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e559706b-fe79-46cf-9998-b0a331522a8c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e559706b-fe79-46cf-9998-b0a331522a8c" (UID: "e559706b-fe79-46cf-9998-b0a331522a8c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:50:45.095226 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.095193    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e559706b-fe79-46cf-9998-b0a331522a8c-kube-api-access-8wvn5" (OuterVolumeSpecName: "kube-api-access-8wvn5") pod "e559706b-fe79-46cf-9998-b0a331522a8c" (UID: "e559706b-fe79-46cf-9998-b0a331522a8c"). InnerVolumeSpecName "kube-api-access-8wvn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:50:45.194959 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.194927    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e559706b-fe79-46cf-9998-b0a331522a8c-extensions-socket-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:50:45.194959 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.194952    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wvn5\" (UniqueName: \"kubernetes.io/projected/e559706b-fe79-46cf-9998-b0a331522a8c-kube-api-access-8wvn5\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:50:45.287088 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.287057    2559 generic.go:356] "Generic (PLEG): container finished" podID="e559706b-fe79-46cf-9998-b0a331522a8c" containerID="aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5" exitCode=0
Aug 04 03:50:45.287214 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.287101    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"
Aug 04 03:50:45.287214 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.287132    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v" event={"ID":"e559706b-fe79-46cf-9998-b0a331522a8c","Type":"ContainerDied","Data":"aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5"}
Aug 04 03:50:45.287214 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.287184    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v" event={"ID":"e559706b-fe79-46cf-9998-b0a331522a8c","Type":"ContainerDied","Data":"0694b4b72cecb4d712083e5fccb6b5cb0ab72e8f58d67f984687be93699abe2b"}
Aug 04 03:50:45.287214 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.287211    2559 scope.go:117] "RemoveContainer" containerID="aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5"
Aug 04 03:50:45.301986 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.301966    2559 scope.go:117] "RemoveContainer" containerID="aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5"
Aug 04 03:50:45.302235 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:50:45.302219    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5\": container with ID starting with aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5 not found: ID does not exist" containerID="aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5"
Aug 04 03:50:45.302289 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.302241    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5"} err="failed to get container status \"aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5\": rpc error: code = NotFound desc = could not find container \"aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5\": container with ID starting with aa55299762132dee1621019913c0a017b89738dc43707ce7d7a977002e0850d5 not found: ID does not exist"
Aug 04 03:50:45.306854 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.306832    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"]
Aug 04 03:50:45.311345 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:45.311326    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4qn6v"]
Aug 04 03:50:47.239624 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:50:47.239590    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e559706b-fe79-46cf-9998-b0a331522a8c" path="/var/lib/kubelet/pods/e559706b-fe79-46cf-9998-b0a331522a8c/volumes"
Aug 04 03:51:13.086606 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:13.086568    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:51:43.416687 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:43.416614    2559 scope.go:117] "RemoveContainer" containerID="168cae2ca18b04ff3bb2bb1f32f1a3b494286061649de106f2590a2e4bce2b10"
Aug 04 03:51:43.424701 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:43.424678    2559 scope.go:117] "RemoveContainer" containerID="3bee1b9fa8cf70f24cbd672d181f2e9902821214740ff7dd6229e333571efd98"
Aug 04 03:51:43.432172 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:43.432154    2559 scope.go:117] "RemoveContainer" containerID="927e2e45d1820fa7aac0f1d90c084a232b2022f7eab9c03790add732fdf39a6d"
Aug 04 03:51:59.764839 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.764805    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"]
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765152    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc52894c-e96c-4fcc-adb6-73031a191cdd" containerName="payload-processing"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765163    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc52894c-e96c-4fcc-adb6-73031a191cdd" containerName="payload-processing"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765172    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerName="istio-proxy"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765177    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerName="istio-proxy"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765193    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5" containerName="payload-pre-processing"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765199    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5" containerName="payload-pre-processing"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765214    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e559706b-fe79-46cf-9998-b0a331522a8c" containerName="manager"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765220    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e559706b-fe79-46cf-9998-b0a331522a8c" containerName="manager"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765284    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecbf1013-d3e9-48b0-ab8d-39b900dd6ac5" containerName="payload-pre-processing"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765293    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerName="istio-proxy"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765299    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerName="istio-proxy"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765305    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e559706b-fe79-46cf-9998-b0a331522a8c" containerName="manager"
Aug 04 03:51:59.765382 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.765314    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc52894c-e96c-4fcc-adb6-73031a191cdd" containerName="payload-processing"
Aug 04 03:51:59.768363 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.768346    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"
Aug 04 03:51:59.770648 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.770627    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rgf8h\""
Aug 04 03:51:59.778424 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.778402    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"]
Aug 04 03:51:59.844015 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.843989    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfl7h\" (UniqueName: \"kubernetes.io/projected/0c243700-e14a-4859-8a81-b058e0bad3fb-kube-api-access-mfl7h\") pod \"kuadrant-operator-controller-manager-86878c769f-2v7ls\" (UID: \"0c243700-e14a-4859-8a81-b058e0bad3fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"
Aug 04 03:51:59.844153 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.844081    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c243700-e14a-4859-8a81-b058e0bad3fb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2v7ls\" (UID: \"0c243700-e14a-4859-8a81-b058e0bad3fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"
Aug 04 03:51:59.944828 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.944794    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c243700-e14a-4859-8a81-b058e0bad3fb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2v7ls\" (UID: \"0c243700-e14a-4859-8a81-b058e0bad3fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"
Aug 04 03:51:59.944960 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.944845    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfl7h\" (UniqueName: \"kubernetes.io/projected/0c243700-e14a-4859-8a81-b058e0bad3fb-kube-api-access-mfl7h\") pod \"kuadrant-operator-controller-manager-86878c769f-2v7ls\" (UID: \"0c243700-e14a-4859-8a81-b058e0bad3fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"
Aug 04 03:51:59.945244 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.945221    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c243700-e14a-4859-8a81-b058e0bad3fb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2v7ls\" (UID: \"0c243700-e14a-4859-8a81-b058e0bad3fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"
Aug 04 03:51:59.954430 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:51:59.954402    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfl7h\" (UniqueName: \"kubernetes.io/projected/0c243700-e14a-4859-8a81-b058e0bad3fb-kube-api-access-mfl7h\") pod \"kuadrant-operator-controller-manager-86878c769f-2v7ls\" (UID: \"0c243700-e14a-4859-8a81-b058e0bad3fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"
Aug 04 03:52:00.078628 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:00.078557    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"
Aug 04 03:52:00.208853 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:00.208828    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"]
Aug 04 03:52:00.210750 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:52:00.210726    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c243700_e14a_4859_8a81_b058e0bad3fb.slice/crio-83ed691ca39faf712835169644910dcc70fdbf744984b39b9690cc55ba975aa4 WatchSource:0}: Error finding container 83ed691ca39faf712835169644910dcc70fdbf744984b39b9690cc55ba975aa4: Status 404 returned error can't find the container with id 83ed691ca39faf712835169644910dcc70fdbf744984b39b9690cc55ba975aa4
Aug 04 03:52:00.652443 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:00.652408    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls" event={"ID":"0c243700-e14a-4859-8a81-b058e0bad3fb","Type":"ContainerStarted","Data":"3eaa37f00b7f9f57d6382568822375b24b351ee22a67a0c67ac49509d5b36717"}
Aug 04 03:52:00.652635 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:00.652456    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls" event={"ID":"0c243700-e14a-4859-8a81-b058e0bad3fb","Type":"ContainerStarted","Data":"83ed691ca39faf712835169644910dcc70fdbf744984b39b9690cc55ba975aa4"}
Aug 04 03:52:00.652635 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:00.652525    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"
Aug 04 03:52:00.673464 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:00.673415    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls" podStartSLOduration=1.673399521 podStartE2EDuration="1.673399521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:51:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:52:00.671919194 +0000 UTC m=+2537.970059682" watchObservedRunningTime="2026-08-04 03:52:00.673399521 +0000 UTC m=+2537.971540007"
Aug 04 03:52:10.714410 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:10.714374    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:10.813490 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:10.813461    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:11.660173 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:11.660142    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2v7ls"
Aug 04 03:52:18.122762 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.122719    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:18.404573 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.404482    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"]
Aug 04 03:52:18.404866 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.404852    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerName="istio-proxy"
Aug 04 03:52:18.404913 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.404867    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a948a631-7e10-47c8-a00a-47c487dd1dc1" containerName="istio-proxy"
Aug 04 03:52:18.407994 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.407979    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.410706 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.410656    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ea8d5546-openshift-default-dockercfg-bfpcn\""
Aug 04 03:52:18.419144 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.419120    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"]
Aug 04 03:52:18.434812 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.434780    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:18.509929 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.509898    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-certs\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.510121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.509936    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7898f10-e6cc-4609-90c8-141d2cad2580-istiod-ca-cert\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.510121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.509957    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-podinfo\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.510121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.510083    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-data\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.510254 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.510183    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-token\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.510254 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.510214    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-socket\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.510254 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.510237    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-credential-socket\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.510381 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.510259    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-envoy\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.510381 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.510308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94gsv\" (UniqueName: \"kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-kube-api-access-94gsv\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.554569 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.554524    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:18.610891 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.610851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-credential-socket\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.611099 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.610895    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-envoy\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.611099 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.610943    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94gsv\" (UniqueName: \"kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-kube-api-access-94gsv\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.611311 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.611268    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-credential-socket\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.611311 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.611273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-certs\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.611499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.611348    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7898f10-e6cc-4609-90c8-141d2cad2580-istiod-ca-cert\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.611499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.611376    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-podinfo\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.611499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.611459    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-data\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.611696 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.611559    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-token\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.611696 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.611574    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-certs\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.611696 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.611587    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-socket\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.611918 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.611872    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-socket\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.612062 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.612039    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-data\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.612557 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.612519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7898f10-e6cc-4609-90c8-141d2cad2580-istiod-ca-cert\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.613802 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.613778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-envoy\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.614125 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.614096    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-podinfo\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.619374 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.619355    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-token\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.619649 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.619627    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94gsv\" (UniqueName: \"kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-kube-api-access-94gsv\") pod \"e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") " pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.718610 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.718494    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:18.855035 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.855010    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"]
Aug 04 03:52:18.856980 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:52:18.856951    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7898f10_e6cc_4609_90c8_141d2cad2580.slice/crio-68e01f13a3aa49926288ad976a888bc8743b9db9f95358659ef87ebc5328865c WatchSource:0}: Error finding container 68e01f13a3aa49926288ad976a888bc8743b9db9f95358659ef87ebc5328865c: Status 404 returned error can't find the container with id 68e01f13a3aa49926288ad976a888bc8743b9db9f95358659ef87ebc5328865c
Aug 04 03:52:18.859055 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.859027    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:52:18.859120 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.859098    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:52:18.859160 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:18.859127    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:52:19.746850 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:19.746811    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw" event={"ID":"e7898f10-e6cc-4609-90c8-141d2cad2580","Type":"ContainerStarted","Data":"28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d"}
Aug 04 03:52:19.747307 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:19.746862    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw" event={"ID":"e7898f10-e6cc-4609-90c8-141d2cad2580","Type":"ContainerStarted","Data":"68e01f13a3aa49926288ad976a888bc8743b9db9f95358659ef87ebc5328865c"}
Aug 04 03:52:19.766094 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:19.766041    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw" podStartSLOduration=1.766025266 podStartE2EDuration="1.766025266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:52:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:52:19.765368555 +0000 UTC m=+2557.063509072" watchObservedRunningTime="2026-08-04 03:52:19.766025266 +0000 UTC m=+2557.064165802"
Aug 04 03:52:20.719495 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:20.719454    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:20.724522 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:20.724488    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:20.751934 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:20.751898    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:20.753015 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:20.752997    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:21.654818 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:21.654774    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:21.746294 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:21.746240    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:22.816710 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:22.816680    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:23.063335 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.063304    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:23.283790 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.283762    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:23.381377 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.381320    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:23.390346 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.390316    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"]
Aug 04 03:52:23.393651 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.393637    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:52:23.395879 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.395858    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-ea8d5546-dockercfg-m8n9z\""
Aug 04 03:52:23.395991 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.395896    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-ea8d5546\""
Aug 04 03:52:23.402998 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.402976    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"]
Aug 04 03:52:23.561828 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.561753    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jvzn\" (UniqueName: \"kubernetes.io/projected/40b68665-7e01-4716-ab57-05f5c8ca8b49-kube-api-access-5jvzn\") pod \"payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47\" (UID: \"40b68665-7e01-4716-ab57-05f5c8ca8b49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:52:23.561828 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.561812    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b68665-7e01-4716-ab57-05f5c8ca8b49-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47\" (UID: \"40b68665-7e01-4716-ab57-05f5c8ca8b49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:52:23.662606 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.662575    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jvzn\" (UniqueName: \"kubernetes.io/projected/40b68665-7e01-4716-ab57-05f5c8ca8b49-kube-api-access-5jvzn\") pod \"payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47\" (UID: \"40b68665-7e01-4716-ab57-05f5c8ca8b49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:52:23.662806 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.662651    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b68665-7e01-4716-ab57-05f5c8ca8b49-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47\" (UID: \"40b68665-7e01-4716-ab57-05f5c8ca8b49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:52:23.663339 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.663320    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b68665-7e01-4716-ab57-05f5c8ca8b49-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47\" (UID: \"40b68665-7e01-4716-ab57-05f5c8ca8b49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:52:23.674860 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.674829    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jvzn\" (UniqueName: \"kubernetes.io/projected/40b68665-7e01-4716-ab57-05f5c8ca8b49-kube-api-access-5jvzn\") pod \"payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47\" (UID: \"40b68665-7e01-4716-ab57-05f5c8ca8b49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:52:23.704504 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:23.704455    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:52:24.044235 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:24.044199    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"]
Aug 04 03:52:24.045422 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:52:24.045398    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b68665_7e01_4716_ab57_05f5c8ca8b49.slice/crio-5415ffd3e6fcd55a844a88c4bc8620de4d74d18c42d9e9f91ba7a7f266917828 WatchSource:0}: Error finding container 5415ffd3e6fcd55a844a88c4bc8620de4d74d18c42d9e9f91ba7a7f266917828: Status 404 returned error can't find the container with id 5415ffd3e6fcd55a844a88c4bc8620de4d74d18c42d9e9f91ba7a7f266917828
Aug 04 03:52:24.778673 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:24.778624    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47" event={"ID":"40b68665-7e01-4716-ab57-05f5c8ca8b49","Type":"ContainerStarted","Data":"f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f"}
Aug 04 03:52:24.778847 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:24.778742    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47" event={"ID":"40b68665-7e01-4716-ab57-05f5c8ca8b49","Type":"ContainerStarted","Data":"5415ffd3e6fcd55a844a88c4bc8620de4d74d18c42d9e9f91ba7a7f266917828"}
Aug 04 03:52:24.779041 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:24.779020    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:52:24.796168 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:24.796126    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47" podStartSLOduration=1.796114161 podStartE2EDuration="1.796114161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:52:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:52:24.795469369 +0000 UTC m=+2562.093609880" watchObservedRunningTime="2026-08-04 03:52:24.796114161 +0000 UTC m=+2562.094254645"
Aug 04 03:52:25.877436 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:25.877405    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:25.929244 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:25.929202    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:34.825834 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:34.825803    2559 generic.go:356] "Generic (PLEG): container finished" podID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerID="28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d" exitCode=0
Aug 04 03:52:34.826274 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:34.825845    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw" event={"ID":"e7898f10-e6cc-4609-90c8-141d2cad2580","Type":"ContainerDied","Data":"28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d"}
Aug 04 03:52:34.826274 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:34.826182    2559 scope.go:117] "RemoveContainer" containerID="28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d"
Aug 04 03:52:34.827133 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:34.827101    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:52:34.827227 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:34.827209    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:52:34.827296 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:34.827261    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:52:35.751961 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:35.751912    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:35.785733 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:35.785711    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:52:35.845565 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:35.842963    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw" event={"ID":"e7898f10-e6cc-4609-90c8-141d2cad2580","Type":"ContainerStarted","Data":"dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0"}
Aug 04 03:52:36.719049 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:36.719011    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:36.720278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:36.720257    2559 patch_prober.go:28] interesting pod/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Aug 04 03:52:36.720365 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:36.720303    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw" podUID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Aug 04 03:52:37.718969 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:37.718932    2559 patch_prober.go:28] interesting pod/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Aug 04 03:52:37.719446 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:37.718992    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw" podUID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Aug 04 03:52:38.719752 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:38.719716    2559 patch_prober.go:28] interesting pod/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Aug 04 03:52:38.720214 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:38.719777    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw" podUID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Aug 04 03:52:39.722902 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:39.722871    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:39.723362 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:39.723184    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:39.723782 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:39.723762    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:52:43.459470 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:43.459441    2559 scope.go:117] "RemoveContainer" containerID="845fee39c2a5c40e3446e7322428cdf397eba5d93a944e8884d6325872689fef"
Aug 04 03:52:43.467215 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:43.467192    2559 scope.go:117] "RemoveContainer" containerID="c1d510650cff14a366a7d687495bd97c6f14fe0381ac5b8311e5865b8ce06114"
Aug 04 03:52:46.617342 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:46.617310    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:49.191804 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.191773    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"]
Aug 04 03:52:49.195109 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.195086    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.197431 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.197401    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ea8d5546\"/\"openshift-service-ca.crt\""
Aug 04 03:52:49.198394 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.198373    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ea8d5546\"/\"e2e-lifecycle-model-ea8d5546-kserve-self-signed-certs\""
Aug 04 03:52:49.198477 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.198443    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ea8d5546\"/\"default-dockercfg-h4gkk\""
Aug 04 03:52:49.198477 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.198461    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ea8d5546\"/\"kube-root-ca.crt\""
Aug 04 03:52:49.203657 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.203632    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"]
Aug 04 03:52:49.212901 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.212880    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:52:49.297611 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.297578    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-model-cache\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.297611 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.297613    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tmp-dir\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.297818 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.297683    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tls-certs\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.297818 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.297731    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbqht\" (UniqueName: \"kubernetes.io/projected/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-kube-api-access-fbqht\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.297818 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.297759    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-home\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.297976 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.297845    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-dshm\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.398754 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.398719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-model-cache\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.398754 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.398754    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tmp-dir\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.399023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.398782    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tls-certs\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.399023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.398802    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbqht\" (UniqueName: \"kubernetes.io/projected/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-kube-api-access-fbqht\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.399023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.398823    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-home\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.399023 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.398854    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-dshm\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.399260 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.399168    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-model-cache\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.399260 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.399191    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-home\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.399374 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.399353    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tmp-dir\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.401113 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.401086    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-dshm\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.401338 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.401323    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tls-certs\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.406361 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.406340    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbqht\" (UniqueName: \"kubernetes.io/projected/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-kube-api-access-fbqht\") pod \"e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") " pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.505745 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.505717    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:49.633576 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.631156    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"]
Aug 04 03:52:49.913116 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.913023    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4" event={"ID":"7a4cd7c4-36f0-4325-9b95-4cab7e4da981","Type":"ContainerStarted","Data":"df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a"}
Aug 04 03:52:49.913116 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.913065    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4" event={"ID":"7a4cd7c4-36f0-4325-9b95-4cab7e4da981","Type":"ContainerStarted","Data":"2a7ce736997b3002304ab91cca30887c59cf6c8ac515e3de211ee2be90258e6e"}
Aug 04 03:52:49.930390 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:49.930337    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4" podStartSLOduration=0.930322315 podStartE2EDuration="930.322315ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:52:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:52:49.92909065 +0000 UTC m=+2587.227231140" watchObservedRunningTime="2026-08-04 03:52:49.930322315 +0000 UTC m=+2587.228462859"
Aug 04 03:52:59.506495 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:59.506459    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:59.506998 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:59.506514    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:59.518935 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:59.518912    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:52:59.971394 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:52:59.971304    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:53:15.685279 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:15.685200    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"]
Aug 04 03:53:15.688956 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:15.688938    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"
Aug 04 03:53:15.691515 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:15.691466    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7lrf4\""
Aug 04 03:53:15.691695 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:15.691532    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 03:53:15.695172 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:15.695149    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"]
Aug 04 03:53:15.748405 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:15.748373    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8e193b-10ca-4909-80d7-79617ad30ff9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5\" (UID: \"bb8e193b-10ca-4909-80d7-79617ad30ff9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"
Aug 04 03:53:15.849487 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:15.849453    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8e193b-10ca-4909-80d7-79617ad30ff9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5\" (UID: \"bb8e193b-10ca-4909-80d7-79617ad30ff9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"
Aug 04 03:53:15.850083 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:15.850063    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8e193b-10ca-4909-80d7-79617ad30ff9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5\" (UID: \"bb8e193b-10ca-4909-80d7-79617ad30ff9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"
Aug 04 03:53:16.001911 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:16.001881    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"
Aug 04 03:53:16.136103 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:16.136080    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"]
Aug 04 03:53:17.043560 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:17.043505    2559 generic.go:356] "Generic (PLEG): container finished" podID="bb8e193b-10ca-4909-80d7-79617ad30ff9" containerID="38188fd5028d73a54a041ce80d798fe745e7c07dc72a9c684733773d75a26446" exitCode=0
Aug 04 03:53:17.043963 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:17.043594    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5" event={"ID":"bb8e193b-10ca-4909-80d7-79617ad30ff9","Type":"ContainerDied","Data":"38188fd5028d73a54a041ce80d798fe745e7c07dc72a9c684733773d75a26446"}
Aug 04 03:53:17.043963 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:17.043654    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5" event={"ID":"bb8e193b-10ca-4909-80d7-79617ad30ff9","Type":"ContainerStarted","Data":"b72044e0b861ff9b6e21f90a400884bfac20edc0a19f4adf863401d0209935c6"}
Aug 04 03:53:18.173300 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:18.173276    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"
Aug 04 03:53:18.264870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:18.264835    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8e193b-10ca-4909-80d7-79617ad30ff9-maas-api-service-ca\") pod \"bb8e193b-10ca-4909-80d7-79617ad30ff9\" (UID: \"bb8e193b-10ca-4909-80d7-79617ad30ff9\") "
Aug 04 03:53:18.265216 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:18.265194    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb8e193b-10ca-4909-80d7-79617ad30ff9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bb8e193b-10ca-4909-80d7-79617ad30ff9" (UID: "bb8e193b-10ca-4909-80d7-79617ad30ff9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:53:18.366066 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:18.365979    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8e193b-10ca-4909-80d7-79617ad30ff9-maas-api-service-ca\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:53:19.054674 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:19.054636    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5" event={"ID":"bb8e193b-10ca-4909-80d7-79617ad30ff9","Type":"ContainerDied","Data":"b72044e0b861ff9b6e21f90a400884bfac20edc0a19f4adf863401d0209935c6"}
Aug 04 03:53:19.054674 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:19.054657    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"
Aug 04 03:53:19.054883 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:19.054684    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b72044e0b861ff9b6e21f90a400884bfac20edc0a19f4adf863401d0209935c6"
Aug 04 03:53:31.312534 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.312471    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:53:31.391889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.391856    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"]
Aug 04 03:53:31.392090 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.392065    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47" podUID="40b68665-7e01-4716-ab57-05f5c8ca8b49" containerName="payload-pre-processing" containerID="cri-o://f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f" gracePeriod=30
Aug 04 03:53:31.439383 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.439351    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:53:31.541750 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.541560    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:53:31.543629 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.543599    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"]
Aug 04 03:53:31.546400 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.546371    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ea8d5546-wqbg5"]
Aug 04 03:53:31.638848 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.638812    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:53:31.664016 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.663988    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:53:31.777571 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.777495    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jvzn\" (UniqueName: \"kubernetes.io/projected/40b68665-7e01-4716-ab57-05f5c8ca8b49-kube-api-access-5jvzn\") pod \"40b68665-7e01-4716-ab57-05f5c8ca8b49\" (UID: \"40b68665-7e01-4716-ab57-05f5c8ca8b49\") "
Aug 04 03:53:31.777766 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.777602    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b68665-7e01-4716-ab57-05f5c8ca8b49-plugins-config-volume\") pod \"40b68665-7e01-4716-ab57-05f5c8ca8b49\" (UID: \"40b68665-7e01-4716-ab57-05f5c8ca8b49\") "
Aug 04 03:53:31.777952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.777927    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40b68665-7e01-4716-ab57-05f5c8ca8b49-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40b68665-7e01-4716-ab57-05f5c8ca8b49" (UID: "40b68665-7e01-4716-ab57-05f5c8ca8b49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:53:31.779504 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.779477    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b68665-7e01-4716-ab57-05f5c8ca8b49-kube-api-access-5jvzn" (OuterVolumeSpecName: "kube-api-access-5jvzn") pod "40b68665-7e01-4716-ab57-05f5c8ca8b49" (UID: "40b68665-7e01-4716-ab57-05f5c8ca8b49"). InnerVolumeSpecName "kube-api-access-5jvzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:53:31.878578 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.878461    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b68665-7e01-4716-ab57-05f5c8ca8b49-plugins-config-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:53:31.878578 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:31.878493    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jvzn\" (UniqueName: \"kubernetes.io/projected/40b68665-7e01-4716-ab57-05f5c8ca8b49-kube-api-access-5jvzn\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:53:32.116909 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:32.116876    2559 generic.go:356] "Generic (PLEG): container finished" podID="40b68665-7e01-4716-ab57-05f5c8ca8b49" containerID="f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f" exitCode=0
Aug 04 03:53:32.117066 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:32.116918    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"
Aug 04 03:53:32.117066 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:32.116956    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47" event={"ID":"40b68665-7e01-4716-ab57-05f5c8ca8b49","Type":"ContainerDied","Data":"f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f"}
Aug 04 03:53:32.117066 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:32.117036    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47" event={"ID":"40b68665-7e01-4716-ab57-05f5c8ca8b49","Type":"ContainerDied","Data":"5415ffd3e6fcd55a844a88c4bc8620de4d74d18c42d9e9f91ba7a7f266917828"}
Aug 04 03:53:32.117173 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:32.117065    2559 scope.go:117] "RemoveContainer" containerID="f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f"
Aug 04 03:53:32.126366 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:32.126347    2559 scope.go:117] "RemoveContainer" containerID="f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f"
Aug 04 03:53:32.126659 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:53:32.126636    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f\": container with ID starting with f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f not found: ID does not exist" containerID="f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f"
Aug 04 03:53:32.126759 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:32.126668    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f"} err="failed to get container status \"f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f\": rpc error: code = NotFound desc = could not find container \"f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f\": container with ID starting with f0a2b3bd7d9b04c2d115b91e0c99c86d2d4e3f8807a0990034b40c90e361372f not found: ID does not exist"
Aug 04 03:53:32.138093 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:32.138035    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"]
Aug 04 03:53:32.141669 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:32.141648    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ea8d5546-5dbbc995d8-kpz47"]
Aug 04 03:53:33.237948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:33.237912    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40b68665-7e01-4716-ab57-05f5c8ca8b49" path="/var/lib/kubelet/pods/40b68665-7e01-4716-ab57-05f5c8ca8b49/volumes"
Aug 04 03:53:33.238318 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:33.238222    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb8e193b-10ca-4909-80d7-79617ad30ff9" path="/var/lib/kubelet/pods/bb8e193b-10ca-4909-80d7-79617ad30ff9/volumes"
Aug 04 03:53:34.546829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:34.546777    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:53:39.534991 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:39.534959    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"]
Aug 04 03:53:39.631414 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:39.631368    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:53:43.487081 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:43.487049    2559 scope.go:117] "RemoveContainer" containerID="0210d4fcc9aeb5069dad8cff954cd4bf81963f33e6a6fcb8df8c1fd4eb2d13e6"
Aug 04 03:53:54.553458 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.553392    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4" podUID="7a4cd7c4-36f0-4325-9b95-4cab7e4da981" containerName="main" containerID="cri-o://df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a" gracePeriod=45
Aug 04 03:53:54.794269 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.794242    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:53:54.881287 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881212    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-dshm\") pod \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") "
Aug 04 03:53:54.881287 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881253    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tmp-dir\") pod \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") "
Aug 04 03:53:54.881509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881314    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tls-certs\") pod \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") "
Aug 04 03:53:54.881509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881359    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-model-cache\") pod \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") "
Aug 04 03:53:54.881509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881383    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-home\") pod \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") "
Aug 04 03:53:54.881509 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881409    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbqht\" (UniqueName: \"kubernetes.io/projected/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-kube-api-access-fbqht\") pod \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\" (UID: \"7a4cd7c4-36f0-4325-9b95-4cab7e4da981\") "
Aug 04 03:53:54.881739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881625    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a4cd7c4-36f0-4325-9b95-4cab7e4da981" (UID: "7a4cd7c4-36f0-4325-9b95-4cab7e4da981"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:53:54.881739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881639    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a4cd7c4-36f0-4325-9b95-4cab7e4da981" (UID: "7a4cd7c4-36f0-4325-9b95-4cab7e4da981"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:53:54.881739 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881682    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-home" (OuterVolumeSpecName: "home") pod "7a4cd7c4-36f0-4325-9b95-4cab7e4da981" (UID: "7a4cd7c4-36f0-4325-9b95-4cab7e4da981"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:53:54.881854 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881773    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-model-cache\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:53:54.881854 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881790    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-home\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:53:54.881854 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.881804    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tmp-dir\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:53:54.883422 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.883390    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a4cd7c4-36f0-4325-9b95-4cab7e4da981" (UID: "7a4cd7c4-36f0-4325-9b95-4cab7e4da981"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 03:53:54.883567 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.883458    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-kube-api-access-fbqht" (OuterVolumeSpecName: "kube-api-access-fbqht") pod "7a4cd7c4-36f0-4325-9b95-4cab7e4da981" (UID: "7a4cd7c4-36f0-4325-9b95-4cab7e4da981"). InnerVolumeSpecName "kube-api-access-fbqht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:53:54.883567 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.883488    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-dshm" (OuterVolumeSpecName: "dshm") pod "7a4cd7c4-36f0-4325-9b95-4cab7e4da981" (UID: "7a4cd7c4-36f0-4325-9b95-4cab7e4da981"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:53:54.982731 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.982694    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-tls-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:53:54.982731 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.982727    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbqht\" (UniqueName: \"kubernetes.io/projected/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-kube-api-access-fbqht\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:53:54.982731 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:54.982740    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a4cd7c4-36f0-4325-9b95-4cab7e4da981-dshm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:53:55.228058 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:55.227971    2559 generic.go:356] "Generic (PLEG): container finished" podID="7a4cd7c4-36f0-4325-9b95-4cab7e4da981" containerID="df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a" exitCode=0
Aug 04 03:53:55.228058 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:55.228009    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"
Aug 04 03:53:55.228058 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:55.228033    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4" event={"ID":"7a4cd7c4-36f0-4325-9b95-4cab7e4da981","Type":"ContainerDied","Data":"df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a"}
Aug 04 03:53:55.228263 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:55.228082    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4" event={"ID":"7a4cd7c4-36f0-4325-9b95-4cab7e4da981","Type":"ContainerDied","Data":"2a7ce736997b3002304ab91cca30887c59cf6c8ac515e3de211ee2be90258e6e"}
Aug 04 03:53:55.228263 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:55.228102    2559 scope.go:117] "RemoveContainer" containerID="df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a"
Aug 04 03:53:55.237380 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:55.237363    2559 scope.go:117] "RemoveContainer" containerID="df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a"
Aug 04 03:53:55.237687 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:53:55.237669    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a\": container with ID starting with df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a not found: ID does not exist" containerID="df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a"
Aug 04 03:53:55.237754 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:55.237696    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a"} err="failed to get container status \"df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a\": rpc error: code = NotFound desc = could not find container \"df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a\": container with ID starting with df7b3effa67cbbc52ab47c3bf6ec6f6f8c7ecd549a93110fdd87d95886493c0a not found: ID does not exist"
Aug 04 03:53:55.248380 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:55.248355    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"]
Aug 04 03:53:55.251765 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:55.251743    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ea8d5546/e2e-lifecycle-model-ea8d5546-kserve-59c97f6f5f-n2wq4"]
Aug 04 03:53:57.242661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:53:57.242626    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a4cd7c4-36f0-4325-9b95-4cab7e4da981" path="/var/lib/kubelet/pods/7a4cd7c4-36f0-4325-9b95-4cab7e4da981/volumes"
Aug 04 03:54:16.352155 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:16.352125    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:17.776483 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:17.776421    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:22.322911 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:22.322855    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:27.895650 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:27.895601    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:27.962099 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:27.962058    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:28.131934 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:28.131900    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"]
Aug 04 03:54:28.132163 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:28.132139    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw" podUID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerName="istio-proxy" containerID="cri-o://dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0" gracePeriod=30
Aug 04 03:54:28.155593 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:28.155488    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:28.363428 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:28.363395    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:33.380532 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.380507    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:54:33.415892 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.415813    2559 generic.go:356] "Generic (PLEG): container finished" podID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerID="dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0" exitCode=0
Aug 04 03:54:33.415892 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.415859    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"
Aug 04 03:54:33.415892 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.415878    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw" event={"ID":"e7898f10-e6cc-4609-90c8-141d2cad2580","Type":"ContainerDied","Data":"dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0"}
Aug 04 03:54:33.416146 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.415918    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw" event={"ID":"e7898f10-e6cc-4609-90c8-141d2cad2580","Type":"ContainerDied","Data":"68e01f13a3aa49926288ad976a888bc8743b9db9f95358659ef87ebc5328865c"}
Aug 04 03:54:33.416146 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.415940    2559 scope.go:117] "RemoveContainer" containerID="dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0"
Aug 04 03:54:33.424087 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.424068    2559 scope.go:117] "RemoveContainer" containerID="28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d"
Aug 04 03:54:33.431512 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.431498    2559 scope.go:117] "RemoveContainer" containerID="dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0"
Aug 04 03:54:33.431733 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:54:33.431716    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0\": container with ID starting with dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0 not found: ID does not exist" containerID="dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0"
Aug 04 03:54:33.431796 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.431743    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0"} err="failed to get container status \"dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0\": rpc error: code = NotFound desc = could not find container \"dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0\": container with ID starting with dade10ea68bee545565bed4ff1c4246523fe0a3c35134f871d2adfb21d4814c0 not found: ID does not exist"
Aug 04 03:54:33.431796 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.431770    2559 scope.go:117] "RemoveContainer" containerID="28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d"
Aug 04 03:54:33.431980 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:54:33.431962    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d\": container with ID starting with 28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d not found: ID does not exist" containerID="28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d"
Aug 04 03:54:33.432020 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.431987    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d"} err="failed to get container status \"28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d\": rpc error: code = NotFound desc = could not find container \"28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d\": container with ID starting with 28da65f282ae398e4401bd1dd1c0b97e77bf25256be39695b457b7e48c54203d not found: ID does not exist"
Aug 04 03:54:33.515195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515164    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-token\") pod \"e7898f10-e6cc-4609-90c8-141d2cad2580\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") "
Aug 04 03:54:33.515357 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515206    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-socket\") pod \"e7898f10-e6cc-4609-90c8-141d2cad2580\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") "
Aug 04 03:54:33.515357 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515229    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-envoy\") pod \"e7898f10-e6cc-4609-90c8-141d2cad2580\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") "
Aug 04 03:54:33.515357 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515246    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-data\") pod \"e7898f10-e6cc-4609-90c8-141d2cad2580\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") "
Aug 04 03:54:33.515357 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515322    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-certs\") pod \"e7898f10-e6cc-4609-90c8-141d2cad2580\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") "
Aug 04 03:54:33.515588 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515374    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-podinfo\") pod \"e7898f10-e6cc-4609-90c8-141d2cad2580\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") "
Aug 04 03:54:33.515588 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515411    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7898f10-e6cc-4609-90c8-141d2cad2580-istiod-ca-cert\") pod \"e7898f10-e6cc-4609-90c8-141d2cad2580\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") "
Aug 04 03:54:33.515588 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515457    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-credential-socket\") pod \"e7898f10-e6cc-4609-90c8-141d2cad2580\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") "
Aug 04 03:54:33.515588 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515489    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94gsv\" (UniqueName: \"kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-kube-api-access-94gsv\") pod \"e7898f10-e6cc-4609-90c8-141d2cad2580\" (UID: \"e7898f10-e6cc-4609-90c8-141d2cad2580\") "
Aug 04 03:54:33.515588 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515525    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e7898f10-e6cc-4609-90c8-141d2cad2580" (UID: "e7898f10-e6cc-4609-90c8-141d2cad2580"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:54:33.515895 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515651    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e7898f10-e6cc-4609-90c8-141d2cad2580" (UID: "e7898f10-e6cc-4609-90c8-141d2cad2580"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:54:33.515895 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515868    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:54:33.515895 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515879    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7898f10-e6cc-4609-90c8-141d2cad2580-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e7898f10-e6cc-4609-90c8-141d2cad2580" (UID: "e7898f10-e6cc-4609-90c8-141d2cad2580"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:54:33.516057 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.515889    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:54:33.516109 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.516082    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e7898f10-e6cc-4609-90c8-141d2cad2580" (UID: "e7898f10-e6cc-4609-90c8-141d2cad2580"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:54:33.516224 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.516199    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-data" (OuterVolumeSpecName: "istio-data") pod "e7898f10-e6cc-4609-90c8-141d2cad2580" (UID: "e7898f10-e6cc-4609-90c8-141d2cad2580"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:54:33.517389 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.517366    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e7898f10-e6cc-4609-90c8-141d2cad2580" (UID: "e7898f10-e6cc-4609-90c8-141d2cad2580"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:54:33.517564 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.517530    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-kube-api-access-94gsv" (OuterVolumeSpecName: "kube-api-access-94gsv") pod "e7898f10-e6cc-4609-90c8-141d2cad2580" (UID: "e7898f10-e6cc-4609-90c8-141d2cad2580"). InnerVolumeSpecName "kube-api-access-94gsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:54:33.517719 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.517697    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7898f10-e6cc-4609-90c8-141d2cad2580" (UID: "e7898f10-e6cc-4609-90c8-141d2cad2580"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:54:33.517820 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.517762    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e7898f10-e6cc-4609-90c8-141d2cad2580" (UID: "e7898f10-e6cc-4609-90c8-141d2cad2580"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 03:54:33.616422 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.616388    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:54:33.616422 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.616414    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:54:33.616422 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.616426    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:54:33.616671 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.616434    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7898f10-e6cc-4609-90c8-141d2cad2580-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:54:33.616671 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.616442    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7898f10-e6cc-4609-90c8-141d2cad2580-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:54:33.616671 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.616452    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94gsv\" (UniqueName: \"kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-kube-api-access-94gsv\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:54:33.616671 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.616461    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7898f10-e6cc-4609-90c8-141d2cad2580-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:54:33.737664 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.737634    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"]
Aug 04 03:54:33.743162 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:33.743138    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ea8d5546-openshift-default-85c4459cd8-25hfw"]
Aug 04 03:54:35.238075 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:35.237998    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7898f10-e6cc-4609-90c8-141d2cad2580" path="/var/lib/kubelet/pods/e7898f10-e6cc-4609-90c8-141d2cad2580/volumes"
Aug 04 03:54:39.586208 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:39.586172    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:39.735014 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:39.734975    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:39.828000 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:39.827963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:40.334259 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:40.334215    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:40.432053 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:40.432026    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:40.535857 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:40.535824    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:41.734177 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:41.734146    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:48.043412 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:48.043371    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:48.143435 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:48.143397    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:48.817497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:48.817461    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:55.068531 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:55.068494    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:55.173911 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:55.173876    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:54:55.270724 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:54:55.270690    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:05.749635 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:05.749602    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:05.963450 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:05.963417    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:26.953460 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:26.953425    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:33.537805 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:33.537756    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:33.641906 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:33.641871    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:33.893861 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:33.893768    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:34.225390 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225318    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"]
Aug 04 03:55:34.225692 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225680    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40b68665-7e01-4716-ab57-05f5c8ca8b49" containerName="payload-pre-processing"
Aug 04 03:55:34.225743 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225694    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="40b68665-7e01-4716-ab57-05f5c8ca8b49" containerName="payload-pre-processing"
Aug 04 03:55:34.225743 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225709    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerName="istio-proxy"
Aug 04 03:55:34.225743 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225718    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerName="istio-proxy"
Aug 04 03:55:34.225743 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225729    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerName="istio-proxy"
Aug 04 03:55:34.225743 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225735    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerName="istio-proxy"
Aug 04 03:55:34.225743 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225745    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a4cd7c4-36f0-4325-9b95-4cab7e4da981" containerName="main"
Aug 04 03:55:34.225920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225750    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a4cd7c4-36f0-4325-9b95-4cab7e4da981" containerName="main"
Aug 04 03:55:34.225920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225756    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb8e193b-10ca-4909-80d7-79617ad30ff9" containerName="revoke-keys"
Aug 04 03:55:34.225920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225761    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb8e193b-10ca-4909-80d7-79617ad30ff9" containerName="revoke-keys"
Aug 04 03:55:34.225920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225819    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="40b68665-7e01-4716-ab57-05f5c8ca8b49" containerName="payload-pre-processing"
Aug 04 03:55:34.225920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225827    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerName="istio-proxy"
Aug 04 03:55:34.225920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225834    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7898f10-e6cc-4609-90c8-141d2cad2580" containerName="istio-proxy"
Aug 04 03:55:34.225920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225839    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb8e193b-10ca-4909-80d7-79617ad30ff9" containerName="revoke-keys"
Aug 04 03:55:34.225920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.225845    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a4cd7c4-36f0-4325-9b95-4cab7e4da981" containerName="main"
Aug 04 03:55:34.228856 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.228828    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.231822 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.231799    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-3a60de-openshift-default-dockercfg-w5r4t\""
Aug 04 03:55:34.245052 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.245030    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"]
Aug 04 03:55:34.259129 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.259108    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:34.361475 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.361442    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-token\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.361475 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.361475    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-credential-socket\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.361742 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.361498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-podinfo\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.361742 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.361558    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-data\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.361742 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.361667    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-certs\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.361742 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.361716    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-socket\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.361742 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.361742    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-envoy\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.361929 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.361770    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istiod-ca-cert\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.361929 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.361858    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npl8d\" (UniqueName: \"kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-kube-api-access-npl8d\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.420945 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.420912    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:34.462907 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.462874    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npl8d\" (UniqueName: \"kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-kube-api-access-npl8d\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463050 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.462912    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-token\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463050 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.462930    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-credential-socket\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463050 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.462975    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-podinfo\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463050 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.463012    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-data\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.463087    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-certs\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.463128    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-socket\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.463149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-envoy\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.463175    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istiod-ca-cert\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463453 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.463297    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-credential-socket\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463510 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.463447    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-socket\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463510 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.463492    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-data\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463632 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.463565    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-certs\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.463918 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.463898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istiod-ca-cert\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.465329 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.465311    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-envoy\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.465439 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.465423    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-podinfo\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.470955 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.470929    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-token\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.471230 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.471209    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npl8d\" (UniqueName: \"kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-kube-api-access-npl8d\") pod \"e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") " pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.539595 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.539534    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:34.670909 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.670885    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"]
Aug 04 03:55:34.672841 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:55:34.672818    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33aba226_f0c5_49ac_bdc4_9ed654f68cc8.slice/crio-7bc6bdb07528a4959af54d5c6d5847641c14ab1441fe5339af9fda0d59d8f210 WatchSource:0}: Error finding container 7bc6bdb07528a4959af54d5c6d5847641c14ab1441fe5339af9fda0d59d8f210: Status 404 returned error can't find the container with id 7bc6bdb07528a4959af54d5c6d5847641c14ab1441fe5339af9fda0d59d8f210
Aug 04 03:55:34.675255 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.675222    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:55:34.675357 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.675306    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:55:34.675419 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.675353    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:55:34.700858 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:34.700835    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" event={"ID":"33aba226-f0c5-49ac-bdc4-9ed654f68cc8","Type":"ContainerStarted","Data":"7bc6bdb07528a4959af54d5c6d5847641c14ab1441fe5339af9fda0d59d8f210"}
Aug 04 03:55:35.708394 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:35.708361    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" event={"ID":"33aba226-f0c5-49ac-bdc4-9ed654f68cc8","Type":"ContainerStarted","Data":"f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3"}
Aug 04 03:55:35.736903 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:35.736861    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" podStartSLOduration=1.7368459189999998 podStartE2EDuration="1.736845919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:55:35.736338991 +0000 UTC m=+2753.034479512" watchObservedRunningTime="2026-08-04 03:55:35.736845919 +0000 UTC m=+2753.034986469"
Aug 04 03:55:36.539914 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:36.539882    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:36.544820 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:36.544796    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:36.713532 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:36.713504    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:36.714485 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:36.714461    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:37.717272 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:37.717242    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:37.805921 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:37.805888    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:38.810185 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:38.810154    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:39.119849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.119765    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:39.400122 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.400042    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:39.602206 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.602176    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:55:39.654574 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.654482    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"]
Aug 04 03:55:39.658089 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.658066    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:55:39.663195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.663176    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-3a60de-dockercfg-zf8p5\""
Aug 04 03:55:39.664927 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.664550    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-3a60de\""
Aug 04 03:55:39.675070 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.675044    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"]
Aug 04 03:55:39.817966 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.817931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03e8aa4b-8c4c-43e1-a4c8-167704487b41-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n\" (UID: \"03e8aa4b-8c4c-43e1-a4c8-167704487b41\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:55:39.818333 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.817995    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjf4l\" (UniqueName: \"kubernetes.io/projected/03e8aa4b-8c4c-43e1-a4c8-167704487b41-kube-api-access-wjf4l\") pod \"payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n\" (UID: \"03e8aa4b-8c4c-43e1-a4c8-167704487b41\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:55:39.919330 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.919233    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjf4l\" (UniqueName: \"kubernetes.io/projected/03e8aa4b-8c4c-43e1-a4c8-167704487b41-kube-api-access-wjf4l\") pod \"payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n\" (UID: \"03e8aa4b-8c4c-43e1-a4c8-167704487b41\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:55:39.919330 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.919321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03e8aa4b-8c4c-43e1-a4c8-167704487b41-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n\" (UID: \"03e8aa4b-8c4c-43e1-a4c8-167704487b41\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:55:39.920000 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.919982    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03e8aa4b-8c4c-43e1-a4c8-167704487b41-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n\" (UID: \"03e8aa4b-8c4c-43e1-a4c8-167704487b41\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:55:39.929141 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.929085    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjf4l\" (UniqueName: \"kubernetes.io/projected/03e8aa4b-8c4c-43e1-a4c8-167704487b41-kube-api-access-wjf4l\") pod \"payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n\" (UID: \"03e8aa4b-8c4c-43e1-a4c8-167704487b41\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:55:39.968952 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:39.968911    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:55:40.103737 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:40.103713    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"]
Aug 04 03:55:40.105377 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:55:40.105349    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e8aa4b_8c4c_43e1_a4c8_167704487b41.slice/crio-4e1c22eb4c6a10237654caa5ee8a1920d3361330cb498ed809d27e6d767506c8 WatchSource:0}: Error finding container 4e1c22eb4c6a10237654caa5ee8a1920d3361330cb498ed809d27e6d767506c8: Status 404 returned error can't find the container with id 4e1c22eb4c6a10237654caa5ee8a1920d3361330cb498ed809d27e6d767506c8
Aug 04 03:55:40.733518 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:40.733479    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n" event={"ID":"03e8aa4b-8c4c-43e1-a4c8-167704487b41","Type":"ContainerStarted","Data":"2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc"}
Aug 04 03:55:40.733742 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:40.733527    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n" event={"ID":"03e8aa4b-8c4c-43e1-a4c8-167704487b41","Type":"ContainerStarted","Data":"4e1c22eb4c6a10237654caa5ee8a1920d3361330cb498ed809d27e6d767506c8"}
Aug 04 03:55:40.733742 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:40.733585    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:55:40.751662 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:40.751609    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n" podStartSLOduration=1.751591598 podStartE2EDuration="1.751591598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:55:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:55:40.751040228 +0000 UTC m=+2758.049180745" watchObservedRunningTime="2026-08-04 03:55:40.751591598 +0000 UTC m=+2758.049732084"
Aug 04 03:55:50.786258 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:50.786226    2559 generic.go:356] "Generic (PLEG): container finished" podID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerID="f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3" exitCode=0
Aug 04 03:55:50.786690 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:50.786302    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" event={"ID":"33aba226-f0c5-49ac-bdc4-9ed654f68cc8","Type":"ContainerDied","Data":"f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3"}
Aug 04 03:55:50.786753 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:50.786701    2559 scope.go:117] "RemoveContainer" containerID="f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3"
Aug 04 03:55:50.787694 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:50.787663    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:55:50.787817 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:50.787786    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:55:50.787889 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:50.787834    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:55:51.714053 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:51.714014    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:51.742816 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:51.742770    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:55:51.792953 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:51.792916    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" event={"ID":"33aba226-f0c5-49ac-bdc4-9ed654f68cc8","Type":"ContainerStarted","Data":"cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c"}
Aug 04 03:55:52.540909 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:52.540873    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:52.542131 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:52.542094    2559 patch_prober.go:28] interesting pod/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Aug 04 03:55:52.542275 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:52.542157    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Aug 04 03:55:53.540458 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:53.540418    2559 patch_prober.go:28] interesting pod/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Aug 04 03:55:53.541068 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:53.541036    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Aug 04 03:55:54.541011 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:54.540974    2559 patch_prober.go:28] interesting pod/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Aug 04 03:55:54.541598 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:54.541564    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Aug 04 03:55:55.544315 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:55.544286    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:55.544764 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:55.544654    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:55:55.545381 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:55:55.545362    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:56:02.687266 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:02.687215    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:03.062063 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.062022    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:03.375796 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.375719    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"]
Aug 04 03:56:03.378219 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.378194    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.387995 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.387975    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e38897-openshift-default-dockercfg-pkpkp\""
Aug 04 03:56:03.396717 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.396692    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"]
Aug 04 03:56:03.406922 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.406893    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:03.433180 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.433147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-certs\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.433337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.433190    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-podinfo\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.433337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.433227    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istiod-ca-cert\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.433337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.433278    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-data\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.433337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.433318    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-socket\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.433471 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.433400    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-token\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.433471 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.433431    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-credential-socket\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.433471 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.433457    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-envoy\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.433615 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.433472    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gjgx\" (UniqueName: \"kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-kube-api-access-8gjgx\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.533970 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.533930    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-data\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.533970 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.533970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-socket\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534018    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-token\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534036    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-credential-socket\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534056    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-envoy\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534163    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gjgx\" (UniqueName: \"kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-kube-api-access-8gjgx\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534435 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534260    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-certs\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534435 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534286    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-podinfo\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534435 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istiod-ca-cert\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534435 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534382    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-data\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534653 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534433    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-credential-socket\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534653 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534486    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-socket\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534774 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534749    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-certs\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.534984 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.534962    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istiod-ca-cert\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.536354 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.536332    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-envoy\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.536660 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.536641    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-podinfo\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.544789 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.544766    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gjgx\" (UniqueName: \"kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-kube-api-access-8gjgx\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.545421 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.545405    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-token\") pod \"e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") " pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.687955 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.687864    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:03.825748 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.825722    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"]
Aug 04 03:56:03.827350 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:56:03.827322    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf14b8d7a_9bc7_451d_a575_74ad1e8dc21f.slice/crio-2e6a0c4acf5b6faf4121be1c94cd16eda8db9fe7ca849c607ba529a6e2f7d653 WatchSource:0}: Error finding container 2e6a0c4acf5b6faf4121be1c94cd16eda8db9fe7ca849c607ba529a6e2f7d653: Status 404 returned error can't find the container with id 2e6a0c4acf5b6faf4121be1c94cd16eda8db9fe7ca849c607ba529a6e2f7d653
Aug 04 03:56:03.829529 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.829483    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:56:03.829666 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.829594    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:56:03.829666 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.829642    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:56:03.850205 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:03.850176    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" event={"ID":"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f","Type":"ContainerStarted","Data":"2e6a0c4acf5b6faf4121be1c94cd16eda8db9fe7ca849c607ba529a6e2f7d653"}
Aug 04 03:56:04.066974 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:04.066938    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:04.856609 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:04.856505    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" event={"ID":"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f","Type":"ContainerStarted","Data":"505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d"}
Aug 04 03:56:04.893780 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:04.893730    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" podStartSLOduration=1.893716685 podStartE2EDuration="1.893716685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:56:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:56:04.892704508 +0000 UTC m=+2782.190844995" watchObservedRunningTime="2026-08-04 03:56:04.893716685 +0000 UTC m=+2782.191857171"
Aug 04 03:56:05.688715 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:05.688686    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:05.693893 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:05.693867    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:05.861216 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:05.861192    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:05.862111 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:05.862093    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:06.971109 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:06.971062    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:07.081669 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:07.081635    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:08.258639 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:08.258595    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:08.501186 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:08.501153    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:08.739010 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:08.738979    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:08.840446 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:08.840407    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:15.283202 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:15.283165    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:21.082077 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:21.081728    2559 patch_prober.go:28] interesting pod/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": read tcp 10.134.0.2:46248->10.134.0.67:15021: read: connection reset by peer" start-of-body=
Aug 04 03:56:21.082077 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:21.081793    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": read tcp 10.134.0.2:46248->10.134.0.67:15021: read: connection reset by peer"
Aug 04 03:56:21.941860 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:21.941775    2559 generic.go:356] "Generic (PLEG): container finished" podID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerID="505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d" exitCode=0
Aug 04 03:56:21.942024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:21.941855    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" event={"ID":"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f","Type":"ContainerDied","Data":"505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d"}
Aug 04 03:56:21.942313 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:21.942297    2559 scope.go:117] "RemoveContainer" containerID="505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d"
Aug 04 03:56:21.943279 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:21.943242    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:56:21.943370 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:21.943330    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:56:21.943430 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:21.943377    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:56:22.951233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:22.951194    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" event={"ID":"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f","Type":"ContainerStarted","Data":"f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d"}
Aug 04 03:56:23.688241 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:23.688203    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:23.688458 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:23.688431    2559 patch_prober.go:28] interesting pod/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Aug 04 03:56:23.688534 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:23.688489    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Aug 04 03:56:24.688829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:24.688799    2559 patch_prober.go:28] interesting pod/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Aug 04 03:56:24.689195 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:24.688847    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Aug 04 03:56:25.688873 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:25.688839    2559 patch_prober.go:28] interesting pod/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Aug 04 03:56:25.689293 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:25.688901    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Aug 04 03:56:26.692460 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:26.692427    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:26.692904 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:26.692816    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:26.693579 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:26.693561    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:56:37.046136 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:37.046105    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:54.885677 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:54.885644    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"]
Aug 04 03:56:54.888610 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:54.888588    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:54.890888 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:54.890870    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3a60de\"/\"kube-root-ca.crt\""
Aug 04 03:56:54.891786 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:54.891765    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3a60de\"/\"default-dockercfg-n2twk\""
Aug 04 03:56:54.891878 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:54.891811    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3a60de\"/\"openshift-service-ca.crt\""
Aug 04 03:56:54.891927 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:54.891775    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3a60de\"/\"test-model-3a60de-kserve-self-signed-certs\""
Aug 04 03:56:54.899390 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:54.899367    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"]
Aug 04 03:56:54.910615 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:54.910588    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:56:55.005771 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.005734    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tls-certs\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.005966 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.005788    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-dshm\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.005966 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.005814    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tmp-dir\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.005966 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.005853    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-model-cache\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.006121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.005966    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghj9g\" (UniqueName: \"kubernetes.io/projected/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-kube-api-access-ghj9g\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.006121 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.005998    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-home\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.106753 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.106717    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghj9g\" (UniqueName: \"kubernetes.io/projected/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-kube-api-access-ghj9g\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.106753 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.106757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-home\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.106998 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.106895    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tls-certs\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.107069 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.107034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-dshm\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.107139 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.107066    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tmp-dir\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.107139 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.107100    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-home\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.107139 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.107115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-model-cache\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.107336 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.107318    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tmp-dir\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.107471 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.107452    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-model-cache\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.109199 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.109176    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-dshm\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.109488 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.109470    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tls-certs\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.115024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.115004    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghj9g\" (UniqueName: \"kubernetes.io/projected/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-kube-api-access-ghj9g\") pod \"test-model-3a60de-kserve-6d9f59cf5-hhvsz\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") " pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.198469 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.198396    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:56:55.332096 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:55.332070    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"]
Aug 04 03:56:55.334204 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:56:55.334166    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5bb0a2fc_337b_4361_a0f1_0bbf75f6c008.slice/crio-cab18dfa1a917d6abbd9f8a79f4e875c47e25e7ba953e3ec199f6f20a85311e0 WatchSource:0}: Error finding container cab18dfa1a917d6abbd9f8a79f4e875c47e25e7ba953e3ec199f6f20a85311e0: Status 404 returned error can't find the container with id cab18dfa1a917d6abbd9f8a79f4e875c47e25e7ba953e3ec199f6f20a85311e0
Aug 04 03:56:56.120185 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:56.120142    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz" event={"ID":"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008","Type":"ContainerStarted","Data":"c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac"}
Aug 04 03:56:56.120581 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:56.120200    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz" event={"ID":"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008","Type":"ContainerStarted","Data":"cab18dfa1a917d6abbd9f8a79f4e875c47e25e7ba953e3ec199f6f20a85311e0"}
Aug 04 03:56:56.137778 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:56:56.137730    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz" podStartSLOduration=2.137713502 podStartE2EDuration="2.137713502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:56:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:56:56.136180498 +0000 UTC m=+2833.434320987" watchObservedRunningTime="2026-08-04 03:56:56.137713502 +0000 UTC m=+2833.435853990"
Aug 04 03:57:05.199616 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:05.199566    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:57:05.199616 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:05.199617    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:57:05.212124 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:05.212093    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:57:06.183393 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:06.183358    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:57:12.466087 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:12.466014    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:57:22.690663 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.690629    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"]
Aug 04 03:57:22.693980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.693964    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.697211 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.697189    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e38897\"/\"default-dockercfg-vccnq\""
Aug 04 03:57:22.697211 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.697201    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e38897\"/\"openshift-service-ca.crt\""
Aug 04 03:57:22.697397 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.697191    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e38897\"/\"kube-root-ca.crt\""
Aug 04 03:57:22.697397 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.697188    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e38897\"/\"test-model-e38897-kserve-self-signed-certs\""
Aug 04 03:57:22.702915 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.702896    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"]
Aug 04 03:57:22.722162 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.722133    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:57:22.769644 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.769611    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-home\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.769644 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.769645    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45021215-e2b5-43e0-bcf4-1dbe67199b57-tls-certs\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.769886 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.769679    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-model-cache\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.769886 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.769757    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-dshm\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.769886 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.769785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-675xt\" (UniqueName: \"kubernetes.io/projected/45021215-e2b5-43e0-bcf4-1dbe67199b57-kube-api-access-675xt\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.769886 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.769809    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-tmp-dir\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.870336 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.870297    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-home\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.870336 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.870333    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45021215-e2b5-43e0-bcf4-1dbe67199b57-tls-certs\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.870619 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.870364    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-model-cache\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.870619 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.870406    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-dshm\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.870619 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.870430    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-675xt\" (UniqueName: \"kubernetes.io/projected/45021215-e2b5-43e0-bcf4-1dbe67199b57-kube-api-access-675xt\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.870619 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.870452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-tmp-dir\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.870899 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.870878    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-home\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.870971 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.870903    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-tmp-dir\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.870971 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.870938    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-model-cache\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.872711 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.872683    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-dshm\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.872886 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.872868    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45021215-e2b5-43e0-bcf4-1dbe67199b57-tls-certs\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:22.879050 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:22.879026    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-675xt\" (UniqueName: \"kubernetes.io/projected/45021215-e2b5-43e0-bcf4-1dbe67199b57-kube-api-access-675xt\") pod \"test-model-e38897-kserve-79f6c776d5-jcqtp\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") " pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:23.004972 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:23.004931    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:23.135503 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:23.135476    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"]
Aug 04 03:57:23.137446 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:57:23.137420    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45021215_e2b5_43e0_bcf4_1dbe67199b57.slice/crio-6bafbc473ce1c7a01652b12e113f58e37745ddccda65e4bcdebe441ee8ed9c46 WatchSource:0}: Error finding container 6bafbc473ce1c7a01652b12e113f58e37745ddccda65e4bcdebe441ee8ed9c46: Status 404 returned error can't find the container with id 6bafbc473ce1c7a01652b12e113f58e37745ddccda65e4bcdebe441ee8ed9c46
Aug 04 03:57:23.259878 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:23.259784    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp" event={"ID":"45021215-e2b5-43e0-bcf4-1dbe67199b57","Type":"ContainerStarted","Data":"c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56"}
Aug 04 03:57:23.259878 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:23.259835    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp" event={"ID":"45021215-e2b5-43e0-bcf4-1dbe67199b57","Type":"ContainerStarted","Data":"6bafbc473ce1c7a01652b12e113f58e37745ddccda65e4bcdebe441ee8ed9c46"}
Aug 04 03:57:23.277938 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:23.277889    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp" podStartSLOduration=1.277874905 podStartE2EDuration="1.277874905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:57:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:57:23.276257938 +0000 UTC m=+2860.574398426" watchObservedRunningTime="2026-08-04 03:57:23.277874905 +0000 UTC m=+2860.576015393"
Aug 04 03:57:33.006085 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:33.006041    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:33.006465 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:33.006108    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:33.018714 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:33.018684    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:57:33.317182 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:57:33.317096    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:58:00.845523 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:00.845436    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:00.899554 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:00.899508    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:06.884080 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:06.884043    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"]
Aug 04 03:58:06.904533 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:06.904502    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:07.779515 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:07.779483    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"]
Aug 04 03:58:07.800250 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:07.800216    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:18.523384 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.523348    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:18.557817 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.557774    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"]
Aug 04 03:58:18.558082 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.558054    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n" podUID="03e8aa4b-8c4c-43e1-a4c8-167704487b41" containerName="payload-pre-processing" containerID="cri-o://2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc" gracePeriod=30
Aug 04 03:58:18.624514 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.624481    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:18.697786 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.697750    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:18.823650 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.823623    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:58:18.892884 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.892846    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03e8aa4b-8c4c-43e1-a4c8-167704487b41-plugins-config-volume\") pod \"03e8aa4b-8c4c-43e1-a4c8-167704487b41\" (UID: \"03e8aa4b-8c4c-43e1-a4c8-167704487b41\") "
Aug 04 03:58:18.892884 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.892884    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjf4l\" (UniqueName: \"kubernetes.io/projected/03e8aa4b-8c4c-43e1-a4c8-167704487b41-kube-api-access-wjf4l\") pod \"03e8aa4b-8c4c-43e1-a4c8-167704487b41\" (UID: \"03e8aa4b-8c4c-43e1-a4c8-167704487b41\") "
Aug 04 03:58:18.893243 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.893219    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03e8aa4b-8c4c-43e1-a4c8-167704487b41-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "03e8aa4b-8c4c-43e1-a4c8-167704487b41" (UID: "03e8aa4b-8c4c-43e1-a4c8-167704487b41"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:58:18.894993 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.894956    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03e8aa4b-8c4c-43e1-a4c8-167704487b41-kube-api-access-wjf4l" (OuterVolumeSpecName: "kube-api-access-wjf4l") pod "03e8aa4b-8c4c-43e1-a4c8-167704487b41" (UID: "03e8aa4b-8c4c-43e1-a4c8-167704487b41"). InnerVolumeSpecName "kube-api-access-wjf4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:58:18.994225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.994190    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03e8aa4b-8c4c-43e1-a4c8-167704487b41-plugins-config-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:18.994225 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:18.994219    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjf4l\" (UniqueName: \"kubernetes.io/projected/03e8aa4b-8c4c-43e1-a4c8-167704487b41-kube-api-access-wjf4l\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:19.082051 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:19.081967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:19.534497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:19.534455    2559 generic.go:356] "Generic (PLEG): container finished" podID="03e8aa4b-8c4c-43e1-a4c8-167704487b41" containerID="2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc" exitCode=0
Aug 04 03:58:19.534497 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:19.534498    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"
Aug 04 03:58:19.535054 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:19.534557    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n" event={"ID":"03e8aa4b-8c4c-43e1-a4c8-167704487b41","Type":"ContainerDied","Data":"2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc"}
Aug 04 03:58:19.535054 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:19.534626    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n" event={"ID":"03e8aa4b-8c4c-43e1-a4c8-167704487b41","Type":"ContainerDied","Data":"4e1c22eb4c6a10237654caa5ee8a1920d3361330cb498ed809d27e6d767506c8"}
Aug 04 03:58:19.535054 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:19.534657    2559 scope.go:117] "RemoveContainer" containerID="2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc"
Aug 04 03:58:19.543956 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:19.543936    2559 scope.go:117] "RemoveContainer" containerID="2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc"
Aug 04 03:58:19.544210 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:58:19.544190    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc\": container with ID starting with 2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc not found: ID does not exist" containerID="2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc"
Aug 04 03:58:19.544263 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:19.544220    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc"} err="failed to get container status \"2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc\": rpc error: code = NotFound desc = could not find container \"2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc\": container with ID starting with 2ec1c85879435b5470816b91a4b6168391e7abd506e6eae1e68cbf3a09c904cc not found: ID does not exist"
Aug 04 03:58:19.552033 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:19.552008    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"]
Aug 04 03:58:19.556037 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:19.556011    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a60de-54445b987b-8kl9n"]
Aug 04 03:58:21.239593 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:21.239557    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03e8aa4b-8c4c-43e1-a4c8-167704487b41" path="/var/lib/kubelet/pods/03e8aa4b-8c4c-43e1-a4c8-167704487b41/volumes"
Aug 04 03:58:21.904318 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:21.904233    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz" podUID="5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" containerName="main" containerID="cri-o://c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac" gracePeriod=45
Aug 04 03:58:22.151135 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.151104    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:58:22.222129 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222027    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tls-certs\") pod \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") "
Aug 04 03:58:22.222129 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222086    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-home\") pod \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") "
Aug 04 03:58:22.222129 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222127    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-model-cache\") pod \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") "
Aug 04 03:58:22.222420 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222198    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-dshm\") pod \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") "
Aug 04 03:58:22.222420 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222227    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghj9g\" (UniqueName: \"kubernetes.io/projected/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-kube-api-access-ghj9g\") pod \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") "
Aug 04 03:58:22.222420 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222252    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tmp-dir\") pod \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\" (UID: \"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008\") "
Aug 04 03:58:22.222420 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222380    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-home" (OuterVolumeSpecName: "home") pod "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" (UID: "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:22.222654 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222496    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" (UID: "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:22.222654 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222513    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-model-cache" (OuterVolumeSpecName: "model-cache") pod "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" (UID: "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:22.222767 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222697    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-home\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:22.222767 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222716    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-model-cache\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:22.222767 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.222757    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tmp-dir\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:22.224357 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.224337    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" (UID: "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 03:58:22.224430 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.224390    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-dshm" (OuterVolumeSpecName: "dshm") pod "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" (UID: "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:22.224472 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.224452    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-kube-api-access-ghj9g" (OuterVolumeSpecName: "kube-api-access-ghj9g") pod "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" (UID: "5bb0a2fc-337b-4361-a0f1-0bbf75f6c008"). InnerVolumeSpecName "kube-api-access-ghj9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:58:22.324145 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.324094    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-tls-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:22.324145 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.324136    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-dshm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:22.324145 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.324151    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghj9g\" (UniqueName: \"kubernetes.io/projected/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008-kube-api-access-ghj9g\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:22.551923 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.551885    2559 generic.go:356] "Generic (PLEG): container finished" podID="5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" containerID="c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac" exitCode=0
Aug 04 03:58:22.552111 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.551943    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"
Aug 04 03:58:22.552111 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.551964    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz" event={"ID":"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008","Type":"ContainerDied","Data":"c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac"}
Aug 04 03:58:22.552111 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.552011    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz" event={"ID":"5bb0a2fc-337b-4361-a0f1-0bbf75f6c008","Type":"ContainerDied","Data":"cab18dfa1a917d6abbd9f8a79f4e875c47e25e7ba953e3ec199f6f20a85311e0"}
Aug 04 03:58:22.552111 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.552032    2559 scope.go:117] "RemoveContainer" containerID="c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac"
Aug 04 03:58:22.561362 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.561343    2559 scope.go:117] "RemoveContainer" containerID="c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac"
Aug 04 03:58:22.561642 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:58:22.561622    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac\": container with ID starting with c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac not found: ID does not exist" containerID="c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac"
Aug 04 03:58:22.561698 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.561649    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac"} err="failed to get container status \"c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac\": rpc error: code = NotFound desc = could not find container \"c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac\": container with ID starting with c1c5b9a1a4928763d748ca2d3d81c72636e9dba0317db607f4f0b7261fdb13ac not found: ID does not exist"
Aug 04 03:58:22.576733 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.576700    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"]
Aug 04 03:58:22.579788 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.579766    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-3a60de/test-model-3a60de-kserve-6d9f59cf5-hhvsz"]
Aug 04 03:58:22.799072 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:22.799026    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp" podUID="45021215-e2b5-43e0-bcf4-1dbe67199b57" containerName="main" containerID="cri-o://c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56" gracePeriod=45
Aug 04 03:58:23.042133 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.042108    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:58:23.131787 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.131697    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-tmp-dir\") pod \"45021215-e2b5-43e0-bcf4-1dbe67199b57\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") "
Aug 04 03:58:23.131787 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.131748    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-675xt\" (UniqueName: \"kubernetes.io/projected/45021215-e2b5-43e0-bcf4-1dbe67199b57-kube-api-access-675xt\") pod \"45021215-e2b5-43e0-bcf4-1dbe67199b57\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") "
Aug 04 03:58:23.132024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.131803    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-home\") pod \"45021215-e2b5-43e0-bcf4-1dbe67199b57\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") "
Aug 04 03:58:23.132024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.131824    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-dshm\") pod \"45021215-e2b5-43e0-bcf4-1dbe67199b57\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") "
Aug 04 03:58:23.132024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.131858    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45021215-e2b5-43e0-bcf4-1dbe67199b57-tls-certs\") pod \"45021215-e2b5-43e0-bcf4-1dbe67199b57\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") "
Aug 04 03:58:23.132024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.131881    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-model-cache\") pod \"45021215-e2b5-43e0-bcf4-1dbe67199b57\" (UID: \"45021215-e2b5-43e0-bcf4-1dbe67199b57\") "
Aug 04 03:58:23.132024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.132002    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "45021215-e2b5-43e0-bcf4-1dbe67199b57" (UID: "45021215-e2b5-43e0-bcf4-1dbe67199b57"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:23.132279 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.132124    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-home" (OuterVolumeSpecName: "home") pod "45021215-e2b5-43e0-bcf4-1dbe67199b57" (UID: "45021215-e2b5-43e0-bcf4-1dbe67199b57"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:23.132279 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.132219    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-model-cache" (OuterVolumeSpecName: "model-cache") pod "45021215-e2b5-43e0-bcf4-1dbe67199b57" (UID: "45021215-e2b5-43e0-bcf4-1dbe67199b57"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:23.132279 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.132243    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-home\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:23.132279 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.132261    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-tmp-dir\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:23.134119 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.134085    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45021215-e2b5-43e0-bcf4-1dbe67199b57-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "45021215-e2b5-43e0-bcf4-1dbe67199b57" (UID: "45021215-e2b5-43e0-bcf4-1dbe67199b57"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 03:58:23.134365 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.134338    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-dshm" (OuterVolumeSpecName: "dshm") pod "45021215-e2b5-43e0-bcf4-1dbe67199b57" (UID: "45021215-e2b5-43e0-bcf4-1dbe67199b57"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:23.134468 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.134361    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45021215-e2b5-43e0-bcf4-1dbe67199b57-kube-api-access-675xt" (OuterVolumeSpecName: "kube-api-access-675xt") pod "45021215-e2b5-43e0-bcf4-1dbe67199b57" (UID: "45021215-e2b5-43e0-bcf4-1dbe67199b57"). InnerVolumeSpecName "kube-api-access-675xt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:58:23.233075 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.233036    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-dshm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:23.233075 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.233073    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45021215-e2b5-43e0-bcf4-1dbe67199b57-tls-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:23.233289 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.233091    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45021215-e2b5-43e0-bcf4-1dbe67199b57-model-cache\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:23.233289 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.233104    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-675xt\" (UniqueName: \"kubernetes.io/projected/45021215-e2b5-43e0-bcf4-1dbe67199b57-kube-api-access-675xt\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:23.238813 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.238774    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" path="/var/lib/kubelet/pods/5bb0a2fc-337b-4361-a0f1-0bbf75f6c008/volumes"
Aug 04 03:58:23.561069 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.561039    2559 generic.go:356] "Generic (PLEG): container finished" podID="45021215-e2b5-43e0-bcf4-1dbe67199b57" containerID="c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56" exitCode=0
Aug 04 03:58:23.561522 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.561081    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"
Aug 04 03:58:23.561522 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.561126    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp" event={"ID":"45021215-e2b5-43e0-bcf4-1dbe67199b57","Type":"ContainerDied","Data":"c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56"}
Aug 04 03:58:23.561522 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.561169    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp" event={"ID":"45021215-e2b5-43e0-bcf4-1dbe67199b57","Type":"ContainerDied","Data":"6bafbc473ce1c7a01652b12e113f58e37745ddccda65e4bcdebe441ee8ed9c46"}
Aug 04 03:58:23.561522 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.561194    2559 scope.go:117] "RemoveContainer" containerID="c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56"
Aug 04 03:58:23.569644 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.569629    2559 scope.go:117] "RemoveContainer" containerID="c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56"
Aug 04 03:58:23.569903 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:58:23.569883    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56\": container with ID starting with c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56 not found: ID does not exist" containerID="c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56"
Aug 04 03:58:23.569956 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.569911    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56"} err="failed to get container status \"c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56\": rpc error: code = NotFound desc = could not find container \"c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56\": container with ID starting with c681303e9cccc9e97642ce149435fbd65060f69f8b21a4da52c8c6410b471e56 not found: ID does not exist"
Aug 04 03:58:23.576119 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.576098    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"]
Aug 04 03:58:23.579318 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:23.579290    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-e38897/test-model-e38897-kserve-79f6c776d5-jcqtp"]
Aug 04 03:58:24.927763 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:24.927728    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:25.028269 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.028216    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:25.189353 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.189256    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"]
Aug 04 03:58:25.189642 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.189612    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerName="istio-proxy" containerID="cri-o://cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c" gracePeriod=30
Aug 04 03:58:25.219007 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.218958    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:25.237831 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.237797    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45021215-e2b5-43e0-bcf4-1dbe67199b57" path="/var/lib/kubelet/pods/45021215-e2b5-43e0-bcf4-1dbe67199b57/volumes"
Aug 04 03:58:25.419075 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.419022    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:25.530646 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.530615    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"]
Aug 04 03:58:25.530980 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.530968    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03e8aa4b-8c4c-43e1-a4c8-167704487b41" containerName="payload-pre-processing"
Aug 04 03:58:25.531026 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.530982    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="03e8aa4b-8c4c-43e1-a4c8-167704487b41" containerName="payload-pre-processing"
Aug 04 03:58:25.531026 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.531000    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45021215-e2b5-43e0-bcf4-1dbe67199b57" containerName="main"
Aug 04 03:58:25.531026 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.531006    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="45021215-e2b5-43e0-bcf4-1dbe67199b57" containerName="main"
Aug 04 03:58:25.531026 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.531015    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" containerName="main"
Aug 04 03:58:25.531026 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.531020    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" containerName="main"
Aug 04 03:58:25.531185 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.531075    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bb0a2fc-337b-4361-a0f1-0bbf75f6c008" containerName="main"
Aug 04 03:58:25.531185 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.531086    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="45021215-e2b5-43e0-bcf4-1dbe67199b57" containerName="main"
Aug 04 03:58:25.531185 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.531092    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="03e8aa4b-8c4c-43e1-a4c8-167704487b41" containerName="payload-pre-processing"
Aug 04 03:58:25.535257 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.535239    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"
Aug 04 03:58:25.538136 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.538111    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 03:58:25.538254 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.538103    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7lrf4\""
Aug 04 03:58:25.541051 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.541028    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"]
Aug 04 03:58:25.546948 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.546920    2559 patch_prober.go:28] interesting pod/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 03:58:25.547072 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.546976    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 03:58:25.657785 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.657743    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01615243-fb66-43de-bcbd-5538b75387c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e38897-dml7g\" (UID: \"01615243-fb66-43de-bcbd-5538b75387c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"
Aug 04 03:58:25.758757 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.758719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01615243-fb66-43de-bcbd-5538b75387c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e38897-dml7g\" (UID: \"01615243-fb66-43de-bcbd-5538b75387c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"
Aug 04 03:58:25.759370 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.759347    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01615243-fb66-43de-bcbd-5538b75387c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e38897-dml7g\" (UID: \"01615243-fb66-43de-bcbd-5538b75387c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"
Aug 04 03:58:25.846672 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.846582    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"
Aug 04 03:58:25.983013 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:25.982986    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"]
Aug 04 03:58:25.985868 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:58:25.985839    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01615243_fb66_43de_bcbd_5538b75387c7.slice/crio-41bca0b49f090dfa428cf6f503090616d88c70dee44bea2b0dab9697a0b46706 WatchSource:0}: Error finding container 41bca0b49f090dfa428cf6f503090616d88c70dee44bea2b0dab9697a0b46706: Status 404 returned error can't find the container with id 41bca0b49f090dfa428cf6f503090616d88c70dee44bea2b0dab9697a0b46706
Aug 04 03:58:26.578924 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:26.578836    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g" event={"ID":"01615243-fb66-43de-bcbd-5538b75387c7","Type":"ContainerStarted","Data":"8499d47b027fc68485d6e92f702817d13b7c569b2a9c539f117ae417431fd2cc"}
Aug 04 03:58:26.578924 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:26.578898    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g" event={"ID":"01615243-fb66-43de-bcbd-5538b75387c7","Type":"ContainerStarted","Data":"41bca0b49f090dfa428cf6f503090616d88c70dee44bea2b0dab9697a0b46706"}
Aug 04 03:58:26.594415 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:26.594363    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g" podStartSLOduration=1.594348807 podStartE2EDuration="1.594348807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:58:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:58:26.59301999 +0000 UTC m=+2923.891166290" watchObservedRunningTime="2026-08-04 03:58:26.594348807 +0000 UTC m=+2923.892489295"
Aug 04 03:58:27.584754 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:27.584722    2559 generic.go:356] "Generic (PLEG): container finished" podID="01615243-fb66-43de-bcbd-5538b75387c7" containerID="8499d47b027fc68485d6e92f702817d13b7c569b2a9c539f117ae417431fd2cc" exitCode=0
Aug 04 03:58:27.585138 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:27.584809    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g" event={"ID":"01615243-fb66-43de-bcbd-5538b75387c7","Type":"ContainerDied","Data":"8499d47b027fc68485d6e92f702817d13b7c569b2a9c539f117ae417431fd2cc"}
Aug 04 03:58:28.723573 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:28.723530    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"
Aug 04 03:58:28.785409 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:28.785380    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01615243-fb66-43de-bcbd-5538b75387c7-maas-api-service-ca\") pod \"01615243-fb66-43de-bcbd-5538b75387c7\" (UID: \"01615243-fb66-43de-bcbd-5538b75387c7\") "
Aug 04 03:58:28.785715 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:28.785694    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01615243-fb66-43de-bcbd-5538b75387c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01615243-fb66-43de-bcbd-5538b75387c7" (UID: "01615243-fb66-43de-bcbd-5538b75387c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:58:28.886345 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:28.886259    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01615243-fb66-43de-bcbd-5538b75387c7-maas-api-service-ca\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:29.596290 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:29.596209    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"
Aug 04 03:58:29.596290 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:29.596230    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g" event={"ID":"01615243-fb66-43de-bcbd-5538b75387c7","Type":"ContainerDied","Data":"41bca0b49f090dfa428cf6f503090616d88c70dee44bea2b0dab9697a0b46706"}
Aug 04 03:58:29.596290 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:29.596273    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41bca0b49f090dfa428cf6f503090616d88c70dee44bea2b0dab9697a0b46706"
Aug 04 03:58:30.448139 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.448113    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:58:30.601814 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.601783    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-certs\") pod \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") "
Aug 04 03:58:30.601968 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.601867    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istiod-ca-cert\") pod \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") "
Aug 04 03:58:30.601968 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.601915    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-socket\") pod \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") "
Aug 04 03:58:30.601968 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.601943    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npl8d\" (UniqueName: \"kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-kube-api-access-npl8d\") pod \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") "
Aug 04 03:58:30.602160 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.601993    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-podinfo\") pod \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") "
Aug 04 03:58:30.602160 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602050    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-token\") pod \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") "
Aug 04 03:58:30.602160 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602076    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33aba226-f0c5-49ac-bdc4-9ed654f68cc8" (UID: "33aba226-f0c5-49ac-bdc4-9ed654f68cc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:30.602160 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602079    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-credential-socket\") pod \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") "
Aug 04 03:58:30.602160 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602134    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-envoy\") pod \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") "
Aug 04 03:58:30.602373 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602173    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-data\") pod \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\" (UID: \"33aba226-f0c5-49ac-bdc4-9ed654f68cc8\") "
Aug 04 03:58:30.602373 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602229    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33aba226-f0c5-49ac-bdc4-9ed654f68cc8" (UID: "33aba226-f0c5-49ac-bdc4-9ed654f68cc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:30.602373 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602255    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33aba226-f0c5-49ac-bdc4-9ed654f68cc8" (UID: "33aba226-f0c5-49ac-bdc4-9ed654f68cc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:58:30.602499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602388    2559 generic.go:356] "Generic (PLEG): container finished" podID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerID="cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c" exitCode=0
Aug 04 03:58:30.602499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602430    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" event={"ID":"33aba226-f0c5-49ac-bdc4-9ed654f68cc8","Type":"ContainerDied","Data":"cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c"}
Aug 04 03:58:30.602499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602483    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t" event={"ID":"33aba226-f0c5-49ac-bdc4-9ed654f68cc8","Type":"ContainerDied","Data":"7bc6bdb07528a4959af54d5c6d5847641c14ab1441fe5339af9fda0d59d8f210"}
Aug 04 03:58:30.602499 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602490    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:30.602706 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602516    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:30.602706 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602533    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:30.602706 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602520    2559 scope.go:117] "RemoveContainer" containerID="cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c"
Aug 04 03:58:30.602858 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602592    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"
Aug 04 03:58:30.602909 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.602650    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33aba226-f0c5-49ac-bdc4-9ed654f68cc8" (UID: "33aba226-f0c5-49ac-bdc4-9ed654f68cc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:30.603242 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.603212    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-data" (OuterVolumeSpecName: "istio-data") pod "33aba226-f0c5-49ac-bdc4-9ed654f68cc8" (UID: "33aba226-f0c5-49ac-bdc4-9ed654f68cc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:30.604676 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.604649    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-token" (OuterVolumeSpecName: "istio-token") pod "33aba226-f0c5-49ac-bdc4-9ed654f68cc8" (UID: "33aba226-f0c5-49ac-bdc4-9ed654f68cc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:58:30.604775 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.604684    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-kube-api-access-npl8d" (OuterVolumeSpecName: "kube-api-access-npl8d") pod "33aba226-f0c5-49ac-bdc4-9ed654f68cc8" (UID: "33aba226-f0c5-49ac-bdc4-9ed654f68cc8"). InnerVolumeSpecName "kube-api-access-npl8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:58:30.604818 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.604767    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33aba226-f0c5-49ac-bdc4-9ed654f68cc8" (UID: "33aba226-f0c5-49ac-bdc4-9ed654f68cc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:30.604818 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.604804    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33aba226-f0c5-49ac-bdc4-9ed654f68cc8" (UID: "33aba226-f0c5-49ac-bdc4-9ed654f68cc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 03:58:30.621983 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.621963    2559 scope.go:117] "RemoveContainer" containerID="f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3"
Aug 04 03:58:30.630024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.629997    2559 scope.go:117] "RemoveContainer" containerID="cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c"
Aug 04 03:58:30.630370 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:58:30.630347    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c\": container with ID starting with cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c not found: ID does not exist" containerID="cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c"
Aug 04 03:58:30.630472 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.630384    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c"} err="failed to get container status \"cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c\": rpc error: code = NotFound desc = could not find container \"cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c\": container with ID starting with cc5a1bc0d2dea7d6d10bd79f1978348e909b5319cf0dd4f590140d46af644f2c not found: ID does not exist"
Aug 04 03:58:30.630472 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.630412    2559 scope.go:117] "RemoveContainer" containerID="f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3"
Aug 04 03:58:30.630735 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:58:30.630701    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3\": container with ID starting with f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3 not found: ID does not exist" containerID="f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3"
Aug 04 03:58:30.630813 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.630733    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3"} err="failed to get container status \"f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3\": rpc error: code = NotFound desc = could not find container \"f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3\": container with ID starting with f4d1a619f2a9ef19bbc14be72281ad4e166f00cddad59b7865061c12c6a15ec3 not found: ID does not exist"
Aug 04 03:58:30.703187 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.703149    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:30.703187 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.703182    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:30.703395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.703213    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npl8d\" (UniqueName: \"kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-kube-api-access-npl8d\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:30.703395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.703223    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:30.703395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.703232    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:30.703395 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.703241    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33aba226-f0c5-49ac-bdc4-9ed654f68cc8-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:30.923959 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.923924    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"]
Aug 04 03:58:30.931252 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:30.931219    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3a60de-openshift-default-f557c9c48-9xw6t"]
Aug 04 03:58:31.237966 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:31.237932    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" path="/var/lib/kubelet/pods/33aba226-f0c5-49ac-bdc4-9ed654f68cc8/volumes"
Aug 04 03:58:35.953648 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:35.953613    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:36.053553 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:36.053509    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:36.119954 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:36.119921    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"]
Aug 04 03:58:36.123086 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:36.123040    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:36.123209 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:36.123130    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e38897-dml7g"]
Aug 04 03:58:36.178710 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:36.178672    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:36.510303 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:36.510268    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:37.237868 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:37.237838    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01615243-fb66-43de-bcbd-5538b75387c7" path="/var/lib/kubelet/pods/01615243-fb66-43de-bcbd-5538b75387c7/volumes"
Aug 04 03:58:42.382308 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:42.382276    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:42.482654 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:42.482619    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:42.627967 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:42.627932    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"]
Aug 04 03:58:42.628182 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:42.628154    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerName="istio-proxy" containerID="cri-o://f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d" gracePeriod=30
Aug 04 03:58:42.649950 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:42.649878    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:42.883482 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:42.883443    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:43.232345 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.232312    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:43.531852 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.531767    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"]
Aug 04 03:58:43.532217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.532151    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01615243-fb66-43de-bcbd-5538b75387c7" containerName="revoke-keys"
Aug 04 03:58:43.532217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.532162    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="01615243-fb66-43de-bcbd-5538b75387c7" containerName="revoke-keys"
Aug 04 03:58:43.532217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.532174    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerName="istio-proxy"
Aug 04 03:58:43.532217 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.532179    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerName="istio-proxy"
Aug 04 03:58:43.532352 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.532239    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="01615243-fb66-43de-bcbd-5538b75387c7" containerName="revoke-keys"
Aug 04 03:58:43.532352 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.532250    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerName="istio-proxy"
Aug 04 03:58:43.532352 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.532258    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerName="istio-proxy"
Aug 04 03:58:43.536670 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.536651    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.538991 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.538970    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ccfdb2-openshift-default-dockercfg-dwsxw\""
Aug 04 03:58:43.549603 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.549579    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"]
Aug 04 03:58:43.555039 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.554996    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:43.721430 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.721399    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbdbw\" (UniqueName: \"kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-kube-api-access-vbdbw\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.721641 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.721439    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-certs\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.721641 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.721462    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-socket\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.721641 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.721478    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-podinfo\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.721641 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.721505    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-token\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.721641 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.721622    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istiod-ca-cert\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.721849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.721669    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-envoy\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.721849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.721719    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-data\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.721849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.721740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-credential-socket\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.750709 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.750673    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:43.822983 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.822890    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istiod-ca-cert\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.822983 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.822943    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-envoy\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.822983 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.822975    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-data\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.823278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.822996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-credential-socket\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.823278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.823061    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbdbw\" (UniqueName: \"kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-kube-api-access-vbdbw\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.823278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.823094    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-certs\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.823278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.823122    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-socket\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.823278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.823150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-podinfo\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.823278 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.823173    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-token\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.823644 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.823440    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-credential-socket\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.823644 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.823530    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-socket\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.823644 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.823584    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-certs\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.823795 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.823661    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-data\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.823795 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.823724    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istiod-ca-cert\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.825186 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.825161    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-envoy\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.825570 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.825535    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-podinfo\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.831022 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.830997    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-token\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.831503 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.831484    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbdbw\" (UniqueName: \"kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-kube-api-access-vbdbw\") pod \"e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") " pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.847330 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.847308    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:43.989902 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.989869    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"]
Aug 04 03:58:43.991079 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:58:43.991047    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ab5b4d_79a7_4757_bf71_6cad83cc850b.slice/crio-cbaf73a75dd90977935344cf4565671f6e77aa5186aa604f611aa0cc0f311e6c WatchSource:0}: Error finding container cbaf73a75dd90977935344cf4565671f6e77aa5186aa604f611aa0cc0f311e6c: Status 404 returned error can't find the container with id cbaf73a75dd90977935344cf4565671f6e77aa5186aa604f611aa0cc0f311e6c
Aug 04 03:58:43.993353 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.993318    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:58:43.993436 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.993385    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:58:43.993477 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:43.993430    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:58:44.675174 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:44.675140    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" event={"ID":"55ab5b4d-79a7-4757-bf71-6cad83cc850b","Type":"ContainerStarted","Data":"c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a"}
Aug 04 03:58:44.675590 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:44.675214    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" event={"ID":"55ab5b4d-79a7-4757-bf71-6cad83cc850b","Type":"ContainerStarted","Data":"cbaf73a75dd90977935344cf4565671f6e77aa5186aa604f611aa0cc0f311e6c"}
Aug 04 03:58:44.693877 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:44.693823    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" podStartSLOduration=1.69380598 podStartE2EDuration="1.69380598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:58:44.693057664 +0000 UTC m=+2941.991198152" watchObservedRunningTime="2026-08-04 03:58:44.69380598 +0000 UTC m=+2941.991946469"
Aug 04 03:58:44.848249 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:44.848219    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:45.853361 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:45.853331    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:46.609455 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:46.609423    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:46.684711 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:46.684686    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:46.685621 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:46.685604    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:58:46.721390 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:46.721357    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:47.806802 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.806776    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:47.881056 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.881028    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:58:47.955279 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955186    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istiod-ca-cert\") pod \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") "
Aug 04 03:58:47.955279 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955229    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-envoy\") pod \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") "
Aug 04 03:58:47.955512 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955281    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-podinfo\") pod \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") "
Aug 04 03:58:47.955512 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955323    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-certs\") pod \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") "
Aug 04 03:58:47.955512 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955346    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-data\") pod \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") "
Aug 04 03:58:47.955512 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955373    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gjgx\" (UniqueName: \"kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-kube-api-access-8gjgx\") pod \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") "
Aug 04 03:58:47.955512 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955424    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-socket\") pod \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") "
Aug 04 03:58:47.955512 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955455    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-token\") pod \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") "
Aug 04 03:58:47.955512 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955490    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-credential-socket\") pod \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\" (UID: \"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f\") "
Aug 04 03:58:47.956233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955597    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" (UID: "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:58:47.956233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955711    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" (UID: "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:47.956233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955780    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" (UID: "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:47.956233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955926    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:47.956233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955947    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:47.956233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955961    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:47.956233 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.955959    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" (UID: "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:47.956479 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.956279    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-data" (OuterVolumeSpecName: "istio-data") pod "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" (UID: "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:47.957660 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.957624    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" (UID: "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 03:58:47.957660 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.957647    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" (UID: "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 03:58:47.957920 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.957903    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-token" (OuterVolumeSpecName: "istio-token") pod "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" (UID: "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:58:47.957968 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:47.957941    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-kube-api-access-8gjgx" (OuterVolumeSpecName: "kube-api-access-8gjgx") pod "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" (UID: "f14b8d7a-9bc7-451d-a575-74ad1e8dc21f"). InnerVolumeSpecName "kube-api-access-8gjgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:58:48.056897 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.056864    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:48.056897 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.056890    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gjgx\" (UniqueName: \"kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-kube-api-access-8gjgx\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:48.056897 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.056900    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:48.057130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.056908    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:48.057130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.056918    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:48.057130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.056929    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:58:48.113388 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.113354    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:48.259996 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.259962    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:48.358183 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.358150    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:58:48.359187 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.359165    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"]
Aug 04 03:58:48.359518 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.359506    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerName="istio-proxy"
Aug 04 03:58:48.359576 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.359520    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerName="istio-proxy"
Aug 04 03:58:48.359576 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.359536    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerName="istio-proxy"
Aug 04 03:58:48.359576 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.359555    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerName="istio-proxy"
Aug 04 03:58:48.359576 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.359570    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerName="istio-proxy"
Aug 04 03:58:48.359576 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.359575    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="33aba226-f0c5-49ac-bdc4-9ed654f68cc8" containerName="istio-proxy"
Aug 04 03:58:48.359738 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.359626    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerName="istio-proxy"
Aug 04 03:58:48.362852 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.362838    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 03:58:48.365291 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.365262    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ccfdb2\""
Aug 04 03:58:48.365412 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.365349    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ccfdb2-dockercfg-6pq9b\""
Aug 04 03:58:48.372847 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.372822    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"]
Aug 04 03:58:48.460138 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.460105    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs\" (UID: \"2f2bdbee-a284-493a-bf4c-18e5c87bbfda\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 03:58:48.460318 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.460198    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4m9d\" (UniqueName: \"kubernetes.io/projected/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-kube-api-access-n4m9d\") pod \"payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs\" (UID: \"2f2bdbee-a284-493a-bf4c-18e5c87bbfda\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 03:58:48.561859 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.561760    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4m9d\" (UniqueName: \"kubernetes.io/projected/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-kube-api-access-n4m9d\") pod \"payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs\" (UID: \"2f2bdbee-a284-493a-bf4c-18e5c87bbfda\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 03:58:48.562033 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.561865    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs\" (UID: \"2f2bdbee-a284-493a-bf4c-18e5c87bbfda\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 03:58:48.562667 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.562648    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs\" (UID: \"2f2bdbee-a284-493a-bf4c-18e5c87bbfda\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 03:58:48.569886 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.569866    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4m9d\" (UniqueName: \"kubernetes.io/projected/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-kube-api-access-n4m9d\") pod \"payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs\" (UID: \"2f2bdbee-a284-493a-bf4c-18e5c87bbfda\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 03:58:48.674015 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.673982    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 03:58:48.697292 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.697259    2559 generic.go:356] "Generic (PLEG): container finished" podID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerID="f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d" exitCode=0
Aug 04 03:58:48.697452 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.697313    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"
Aug 04 03:58:48.697452 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.697340    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" event={"ID":"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f","Type":"ContainerDied","Data":"f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d"}
Aug 04 03:58:48.697452 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.697397    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg" event={"ID":"f14b8d7a-9bc7-451d-a575-74ad1e8dc21f","Type":"ContainerDied","Data":"2e6a0c4acf5b6faf4121be1c94cd16eda8db9fe7ca849c607ba529a6e2f7d653"}
Aug 04 03:58:48.697452 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.697422    2559 scope.go:117] "RemoveContainer" containerID="f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d"
Aug 04 03:58:48.714524 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.714499    2559 scope.go:117] "RemoveContainer" containerID="505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d"
Aug 04 03:58:48.725373 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.725094    2559 scope.go:117] "RemoveContainer" containerID="f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d"
Aug 04 03:58:48.726117 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:58:48.726067    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d\": container with ID starting with f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d not found: ID does not exist" containerID="f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d"
Aug 04 03:58:48.726231 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.726128    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d"} err="failed to get container status \"f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d\": rpc error: code = NotFound desc = could not find container \"f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d\": container with ID starting with f997c8dd1ba66a6ba6eac2a15e91e662d16c41f3c4ac75206747cec4d665b00d not found: ID does not exist"
Aug 04 03:58:48.726231 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.726155    2559 scope.go:117] "RemoveContainer" containerID="505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d"
Aug 04 03:58:48.726509 ip-10-0-134-110 kubenswrapper[2559]: E0804 03:58:48.726488    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d\": container with ID starting with 505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d not found: ID does not exist" containerID="505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d"
Aug 04 03:58:48.726637 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.726518    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d"} err="failed to get container status \"505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d\": rpc error: code = NotFound desc = could not find container \"505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d\": container with ID starting with 505bc7ec8fc0b76598dfdbbb88590ce6d2d99d5cab9408caa6ec793fba83a67d not found: ID does not exist"
Aug 04 03:58:48.727849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.727769    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"]
Aug 04 03:58:48.735359 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.734693    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e38897-openshift-default-fd75b46cb-npcrg"]
Aug 04 03:58:48.815494 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:48.815472    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"]
Aug 04 03:58:48.817111 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:58:48.817087    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2bdbee_a284_493a_bf4c_18e5c87bbfda.slice/crio-2cb19b7ad8fe4d7bba81ec75da0d84f1beeaa8d875c54163172ebea0c644e151 WatchSource:0}: Error finding container 2cb19b7ad8fe4d7bba81ec75da0d84f1beeaa8d875c54163172ebea0c644e151: Status 404 returned error can't find the container with id 2cb19b7ad8fe4d7bba81ec75da0d84f1beeaa8d875c54163172ebea0c644e151
Aug 04 03:58:49.240116 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:49.240078    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" path="/var/lib/kubelet/pods/f14b8d7a-9bc7-451d-a575-74ad1e8dc21f/volumes"
Aug 04 03:58:49.703784 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:49.703690    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs" event={"ID":"2f2bdbee-a284-493a-bf4c-18e5c87bbfda","Type":"ContainerStarted","Data":"21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176"}
Aug 04 03:58:49.703784 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:49.703752    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs" event={"ID":"2f2bdbee-a284-493a-bf4c-18e5c87bbfda","Type":"ContainerStarted","Data":"2cb19b7ad8fe4d7bba81ec75da0d84f1beeaa8d875c54163172ebea0c644e151"}
Aug 04 03:58:49.703784 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:49.703783    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 03:58:49.718586 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:58:49.718523    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs" podStartSLOduration=1.718508406 podStartE2EDuration="1.718508406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:58:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:58:49.717101425 +0000 UTC m=+2947.015241930" watchObservedRunningTime="2026-08-04 03:58:49.718508406 +0000 UTC m=+2947.016648938"
Aug 04 03:59:00.711740 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:00.711701    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 03:59:01.732669 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:01.731359    2559 patch_prober.go:28] interesting pod/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": read tcp 10.134.0.2:60974->10.134.0.71:15021: read: connection reset by peer" start-of-body=
Aug 04 03:59:01.732669 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:01.731443    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": read tcp 10.134.0.2:60974->10.134.0.71:15021: read: connection reset by peer"
Aug 04 03:59:01.773711 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:01.773677    2559 generic.go:356] "Generic (PLEG): container finished" podID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerID="c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a" exitCode=0
Aug 04 03:59:01.773870 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:01.773743    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" event={"ID":"55ab5b4d-79a7-4757-bf71-6cad83cc850b","Type":"ContainerDied","Data":"c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a"}
Aug 04 03:59:01.774182 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:01.774166    2559 scope.go:117] "RemoveContainer" containerID="c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a"
Aug 04 03:59:01.775188 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:01.775160    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:59:01.775288 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:01.775242    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:59:01.775362 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:01.775290    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:59:02.780293 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:02.780258    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" event={"ID":"55ab5b4d-79a7-4757-bf71-6cad83cc850b","Type":"ContainerStarted","Data":"961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c"}
Aug 04 03:59:02.848616 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:02.848575    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:59:02.848831 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:02.848724    2559 patch_prober.go:28] interesting pod/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Aug 04 03:59:02.848831 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:02.848770    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Aug 04 03:59:03.848592 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:03.848525    2559 patch_prober.go:28] interesting pod/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Aug 04 03:59:03.849104 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:03.848619    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Aug 04 03:59:04.848526 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:04.848425    2559 patch_prober.go:28] interesting pod/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Aug 04 03:59:04.848849 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:04.848815    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Aug 04 03:59:05.848680 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:05.848197    2559 patch_prober.go:28] interesting pod/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Aug 04 03:59:05.848680 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:05.848262    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Aug 04 03:59:06.851184 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:06.851156    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:59:06.851861 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:06.851417    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:59:06.852054 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:06.852040    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 03:59:11.586819 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:11.586778    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:12.059347 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.059290    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:12.360483 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.360393    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"]
Aug 04 03:59:12.361070 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.361048    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f14b8d7a-9bc7-451d-a575-74ad1e8dc21f" containerName="istio-proxy"
Aug 04 03:59:12.364672 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.364644    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.366834 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.366815    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-523226-openshift-default-dockercfg-sbgzg\""
Aug 04 03:59:12.383005 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.382970    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"]
Aug 04 03:59:12.391238 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.391211    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:12.480937 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.480900    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22vw4\" (UniqueName: \"kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-kube-api-access-22vw4\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.481134 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.480945    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-credential-socket\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.481134 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.480979    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-socket\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.481134 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.481002    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-data\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.481134 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.481023    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istiod-ca-cert\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.481134 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.481081    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-podinfo\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.481434 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.481135    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-certs\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.481434 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.481153    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-envoy\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.481434 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.481257    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-token\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.529039 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.529003    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:12.581895 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.581856    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-token\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.581952    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22vw4\" (UniqueName: \"kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-kube-api-access-22vw4\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.581980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-credential-socket\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.582000    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-socket\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.582024    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-data\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.582048    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istiod-ca-cert\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.582080    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-podinfo\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582130 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.582124    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-certs\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582487 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.582150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-envoy\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.582478    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-credential-socket\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.582503    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-socket\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582580 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.582569    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-data\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582707 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.582633    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-certs\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.582855 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.582837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istiod-ca-cert\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.584379 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.584360    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-envoy\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.584629 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.584609    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-podinfo\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.589259 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.589230    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-token\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.590386 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.590344    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22vw4\" (UniqueName: \"kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-kube-api-access-22vw4\") pod \"e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") " pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.677645 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.677490    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:12.820638 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.820615    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"]
Aug 04 03:59:12.822325 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:59:12.822292    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c58890a_a3b3_4ea8_8cd7_b4b08b23e3b7.slice/crio-19820ad0f44a5247622ec60b559e54f3c58159070bee17d0c93dea223fd615e8 WatchSource:0}: Error finding container 19820ad0f44a5247622ec60b559e54f3c58159070bee17d0c93dea223fd615e8: Status 404 returned error can't find the container with id 19820ad0f44a5247622ec60b559e54f3c58159070bee17d0c93dea223fd615e8
Aug 04 03:59:12.824493 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.824447    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:59:12.824618 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.824556    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:59:12.824618 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.824597    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:59:12.830590 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:12.830566    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" event={"ID":"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7","Type":"ContainerStarted","Data":"19820ad0f44a5247622ec60b559e54f3c58159070bee17d0c93dea223fd615e8"}
Aug 04 03:59:13.838603 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:13.838565    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" event={"ID":"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7","Type":"ContainerStarted","Data":"6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1"}
Aug 04 03:59:13.864897 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:13.864833    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" podStartSLOduration=1.8648135510000001 podStartE2EDuration="1.864813551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 03:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 03:59:13.862261691 +0000 UTC m=+2971.160402204" watchObservedRunningTime="2026-08-04 03:59:13.864813551 +0000 UTC m=+2971.162954040"
Aug 04 03:59:14.677822 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:14.677785    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:14.682829 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:14.682802    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:14.844169 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:14.844139    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:14.845364 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:14.845337    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:15.827350 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:15.827318    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:15.951024 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:15.950992    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:17.002589 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:16.998358    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:17.106032 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:17.105998    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:17.454025 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:17.453898    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:17.547679 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:17.547626    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:28.915814 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:28.915739    2559 generic.go:356] "Generic (PLEG): container finished" podID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerID="6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1" exitCode=0
Aug 04 03:59:28.915814 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:28.915781    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" event={"ID":"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7","Type":"ContainerDied","Data":"6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1"}
Aug 04 03:59:28.916287 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:28.916209    2559 scope.go:117] "RemoveContainer" containerID="6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1"
Aug 04 03:59:28.917303 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:28.917268    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:59:28.917378 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:28.917360    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:59:28.917417 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:28.917403    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 03:59:29.845086 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:29.844989    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:29.922179 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:29.922148    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" event={"ID":"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7","Type":"ContainerStarted","Data":"cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983"}
Aug 04 03:59:30.677623 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:30.677586    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:30.678683 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:30.678657    2559 patch_prober.go:28] interesting pod/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Aug 04 03:59:30.678811 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:30.678709    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Aug 04 03:59:31.678239 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:31.678203    2559 patch_prober.go:28] interesting pod/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Aug 04 03:59:31.678706 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:31.678269    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Aug 04 03:59:32.678669 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:32.678632    2559 patch_prober.go:28] interesting pod/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Aug 04 03:59:32.679063 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:32.678691    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Aug 04 03:59:33.678455 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:33.678417    2559 patch_prober.go:28] interesting pod/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Aug 04 03:59:33.678671 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:33.678488    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Aug 04 03:59:34.682689 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:34.682654    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:34.683136 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:34.682926    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:34.683710 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:34.683690    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 03:59:40.761901 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:40.761860    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:43.577276 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:43.577240    2559 scope.go:117] "RemoveContainer" containerID="38188fd5028d73a54a041ce80d798fe745e7c07dc72a9c684733773d75a26446"
Aug 04 03:59:43.673181 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:43.673147    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"]
Aug 04 03:59:43.678042 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:43.678024    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"
Aug 04 03:59:43.680496 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:43.680473    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 03:59:43.681109 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:43.681089    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7lrf4\""
Aug 04 03:59:43.683505 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:43.683482    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"]
Aug 04 03:59:43.781767 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:43.781739    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0389a920-f2d3-4b07-a362-886137184aa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp\" (UID: \"0389a920-f2d3-4b07-a362-886137184aa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"
Aug 04 03:59:43.882873 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:43.882787    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0389a920-f2d3-4b07-a362-886137184aa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp\" (UID: \"0389a920-f2d3-4b07-a362-886137184aa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"
Aug 04 03:59:43.883437 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:43.883415    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0389a920-f2d3-4b07-a362-886137184aa0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp\" (UID: \"0389a920-f2d3-4b07-a362-886137184aa0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"
Aug 04 03:59:43.989731 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:43.989699    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"
Aug 04 03:59:44.133275 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:44.133204    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"]
Aug 04 03:59:44.135077 ip-10-0-134-110 kubenswrapper[2559]: W0804 03:59:44.135048    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0389a920_f2d3_4b07_a362_886137184aa0.slice/crio-5bdc687500a091f4c2e4445588b733965bb8a232d5769cb8c4a19945b4f14044 WatchSource:0}: Error finding container 5bdc687500a091f4c2e4445588b733965bb8a232d5769cb8c4a19945b4f14044: Status 404 returned error can't find the container with id 5bdc687500a091f4c2e4445588b733965bb8a232d5769cb8c4a19945b4f14044
Aug 04 03:59:45.004661 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:45.004622    2559 generic.go:356] "Generic (PLEG): container finished" podID="0389a920-f2d3-4b07-a362-886137184aa0" containerID="957b6416ea2eb3ec44b247c10f7bf5a96f5d18dcced97c28230b24cb22c5823f" exitCode=0
Aug 04 03:59:45.005053 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:45.004720    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp" event={"ID":"0389a920-f2d3-4b07-a362-886137184aa0","Type":"ContainerDied","Data":"957b6416ea2eb3ec44b247c10f7bf5a96f5d18dcced97c28230b24cb22c5823f"}
Aug 04 03:59:45.005053 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:45.004777    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp" event={"ID":"0389a920-f2d3-4b07-a362-886137184aa0","Type":"ContainerStarted","Data":"5bdc687500a091f4c2e4445588b733965bb8a232d5769cb8c4a19945b4f14044"}
Aug 04 03:59:46.142611 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:46.142574    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"
Aug 04 03:59:46.204290 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:46.204247    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0389a920-f2d3-4b07-a362-886137184aa0-maas-api-service-ca\") pod \"0389a920-f2d3-4b07-a362-886137184aa0\" (UID: \"0389a920-f2d3-4b07-a362-886137184aa0\") "
Aug 04 03:59:46.204634 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:46.204612    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0389a920-f2d3-4b07-a362-886137184aa0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0389a920-f2d3-4b07-a362-886137184aa0" (UID: "0389a920-f2d3-4b07-a362-886137184aa0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:59:46.305337 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:46.305255    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0389a920-f2d3-4b07-a362-886137184aa0-maas-api-service-ca\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:59:47.015855 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:47.015821    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp" event={"ID":"0389a920-f2d3-4b07-a362-886137184aa0","Type":"ContainerDied","Data":"5bdc687500a091f4c2e4445588b733965bb8a232d5769cb8c4a19945b4f14044"}
Aug 04 03:59:47.016033 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:47.015877    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bdc687500a091f4c2e4445588b733965bb8a232d5769cb8c4a19945b4f14044"
Aug 04 03:59:47.016033 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:47.015830    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"
Aug 04 03:59:52.316978 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:52.316915    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:52.417536 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:52.417503    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:59.303137 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.303099    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:59.335762 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.335729    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"]
Aug 04 03:59:59.336002 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.335971    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs" podUID="2f2bdbee-a284-493a-bf4c-18e5c87bbfda" containerName="payload-pre-processing" containerID="cri-o://21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176" gracePeriod=30
Aug 04 03:59:59.403689 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.403651    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:59.504587 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.504536    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:59.521221 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.521186    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"]
Aug 04 03:59:59.525182 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.525088    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ccfdb2-9r2gp"]
Aug 04 03:59:59.573414 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.573310    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 03:59:59.601385 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.601362    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 03:59:59.737256 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.737224    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-plugins-config-volume\") pod \"2f2bdbee-a284-493a-bf4c-18e5c87bbfda\" (UID: \"2f2bdbee-a284-493a-bf4c-18e5c87bbfda\") "
Aug 04 03:59:59.737445 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.737305    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4m9d\" (UniqueName: \"kubernetes.io/projected/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-kube-api-access-n4m9d\") pod \"2f2bdbee-a284-493a-bf4c-18e5c87bbfda\" (UID: \"2f2bdbee-a284-493a-bf4c-18e5c87bbfda\") "
Aug 04 03:59:59.737632 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.737608    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f2bdbee-a284-493a-bf4c-18e5c87bbfda" (UID: "2f2bdbee-a284-493a-bf4c-18e5c87bbfda"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 03:59:59.739386 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.739364    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-kube-api-access-n4m9d" (OuterVolumeSpecName: "kube-api-access-n4m9d") pod "2f2bdbee-a284-493a-bf4c-18e5c87bbfda" (UID: "2f2bdbee-a284-493a-bf4c-18e5c87bbfda"). InnerVolumeSpecName "kube-api-access-n4m9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 03:59:59.838645 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.838513    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-plugins-config-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:59:59.838645 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.838584    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4m9d\" (UniqueName: \"kubernetes.io/projected/2f2bdbee-a284-493a-bf4c-18e5c87bbfda-kube-api-access-n4m9d\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 03:59:59.889955 ip-10-0-134-110 kubenswrapper[2559]: I0804 03:59:59.889924    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:00.085358 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.085326    2559 generic.go:356] "Generic (PLEG): container finished" podID="2f2bdbee-a284-493a-bf4c-18e5c87bbfda" containerID="21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176" exitCode=0
Aug 04 04:00:00.085508 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.085371    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"
Aug 04 04:00:00.085508 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.085399    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs" event={"ID":"2f2bdbee-a284-493a-bf4c-18e5c87bbfda","Type":"ContainerDied","Data":"21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176"}
Aug 04 04:00:00.085508 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.085446    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs" event={"ID":"2f2bdbee-a284-493a-bf4c-18e5c87bbfda","Type":"ContainerDied","Data":"2cb19b7ad8fe4d7bba81ec75da0d84f1beeaa8d875c54163172ebea0c644e151"}
Aug 04 04:00:00.085508 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.085470    2559 scope.go:117] "RemoveContainer" containerID="21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176"
Aug 04 04:00:00.094670 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.094650    2559 scope.go:117] "RemoveContainer" containerID="21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176"
Aug 04 04:00:00.094950 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:00:00.094928    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176\": container with ID starting with 21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176 not found: ID does not exist" containerID="21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176"
Aug 04 04:00:00.095044 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.094957    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176"} err="failed to get container status \"21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176\": rpc error: code = NotFound desc = could not find container \"21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176\": container with ID starting with 21afd5b66ffc5deab0b488cb3908a2532e39108e4012f4b6c783399eefc52176 not found: ID does not exist"
Aug 04 04:00:00.110411 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.110370    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"]
Aug 04 04:00:00.112883 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.112856    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ccfdb2-7b556f84d8-hvwhs"]
Aug 04 04:00:00.147119 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.147089    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk"]
Aug 04 04:00:00.147467 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.147454    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0389a920-f2d3-4b07-a362-886137184aa0" containerName="revoke-keys"
Aug 04 04:00:00.147516 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.147468    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0389a920-f2d3-4b07-a362-886137184aa0" containerName="revoke-keys"
Aug 04 04:00:00.147516 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.147486    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f2bdbee-a284-493a-bf4c-18e5c87bbfda" containerName="payload-pre-processing"
Aug 04 04:00:00.147516 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.147491    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f2bdbee-a284-493a-bf4c-18e5c87bbfda" containerName="payload-pre-processing"
Aug 04 04:00:00.147649 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.147557    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0389a920-f2d3-4b07-a362-886137184aa0" containerName="revoke-keys"
Aug 04 04:00:00.147649 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.147565    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f2bdbee-a284-493a-bf4c-18e5c87bbfda" containerName="payload-pre-processing"
Aug 04 04:00:00.151968 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.151944    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk"
Aug 04 04:00:00.158290 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.158263    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk"]
Aug 04 04:00:00.166407 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.166380    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7lrf4\""
Aug 04 04:00:00.173399 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.173328    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk"
Aug 04 04:00:00.307871 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:00.307705    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk"]
Aug 04 04:00:00.310122 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:00:00.310090    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf5487a0_7de8_43da_ae01_9d6ba493632d.slice/crio-7f03e9df925e5f964cd3e5ab7587fb91fb52835d9772f548600b0399931efbae WatchSource:0}: Error finding container 7f03e9df925e5f964cd3e5ab7587fb91fb52835d9772f548600b0399931efbae: Status 404 returned error can't find the container with id 7f03e9df925e5f964cd3e5ab7587fb91fb52835d9772f548600b0399931efbae
Aug 04 04:00:01.092853 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:01.092817    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk" event={"ID":"df5487a0-7de8-43da-ae01-9d6ba493632d","Type":"ContainerStarted","Data":"142e5ade65b31da41ad028c3cab0f0c0f665cf97e7949c3810f3e4007f01f499"}
Aug 04 04:00:01.093044 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:01.092883    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk" event={"ID":"df5487a0-7de8-43da-ae01-9d6ba493632d","Type":"ContainerStarted","Data":"7f03e9df925e5f964cd3e5ab7587fb91fb52835d9772f548600b0399931efbae"}
Aug 04 04:00:01.109565 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:01.109488    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk" podStartSLOduration=1.109470567 podStartE2EDuration="1.109470567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:00:01.107723487 +0000 UTC m=+3018.405863974" watchObservedRunningTime="2026-08-04 04:00:01.109470567 +0000 UTC m=+3018.407611077"
Aug 04 04:00:01.240428 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:01.240391    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0389a920-f2d3-4b07-a362-886137184aa0" path="/var/lib/kubelet/pods/0389a920-f2d3-4b07-a362-886137184aa0/volumes"
Aug 04 04:00:01.240978 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:01.240956    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f2bdbee-a284-493a-bf4c-18e5c87bbfda" path="/var/lib/kubelet/pods/2f2bdbee-a284-493a-bf4c-18e5c87bbfda/volumes"
Aug 04 04:00:02.100443 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:02.100407    2559 generic.go:356] "Generic (PLEG): container finished" podID="df5487a0-7de8-43da-ae01-9d6ba493632d" containerID="142e5ade65b31da41ad028c3cab0f0c0f665cf97e7949c3810f3e4007f01f499" exitCode=0
Aug 04 04:00:02.100865 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:02.100476    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk" event={"ID":"df5487a0-7de8-43da-ae01-9d6ba493632d","Type":"ContainerDied","Data":"142e5ade65b31da41ad028c3cab0f0c0f665cf97e7949c3810f3e4007f01f499"}
Aug 04 04:00:03.246053 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:03.246021    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk"
Aug 04 04:00:04.110874 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:04.110841    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk" event={"ID":"df5487a0-7de8-43da-ae01-9d6ba493632d","Type":"ContainerDied","Data":"7f03e9df925e5f964cd3e5ab7587fb91fb52835d9772f548600b0399931efbae"}
Aug 04 04:00:04.110874 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:04.110871    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29763600-ckcvk"
Aug 04 04:00:04.111088 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:04.110912    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f03e9df925e5f964cd3e5ab7587fb91fb52835d9772f548600b0399931efbae"
Aug 04 04:00:05.614400 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:05.614371    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:05.863078 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:05.863047    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"]
Aug 04 04:00:05.863268 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:05.863246    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy" containerID="cri-o://961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c" gracePeriod=30
Aug 04 04:00:05.864150 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:05.864129    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:05.928317 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:05.928203    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:06.095207 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.095159    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:06.214074 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.213995    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"]
Aug 04 04:00:06.214363 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.214351    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df5487a0-7de8-43da-ae01-9d6ba493632d" containerName="cleanup"
Aug 04 04:00:06.214411 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.214365    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df5487a0-7de8-43da-ae01-9d6ba493632d" containerName="cleanup"
Aug 04 04:00:06.214446 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.214428    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="df5487a0-7de8-43da-ae01-9d6ba493632d" containerName="cleanup"
Aug 04 04:00:06.217615 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.217597    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"
Aug 04 04:00:06.219885 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.219864    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7lrf4\""
Aug 04 04:00:06.220012 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.219866    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 04:00:06.225689 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.225666    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"]
Aug 04 04:00:06.296053 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.296020    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-523226-9wnlb\" (UID: \"ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"
Aug 04 04:00:06.397123 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.397087    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-523226-9wnlb\" (UID: \"ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"
Aug 04 04:00:06.397744 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.397723    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-523226-9wnlb\" (UID: \"ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"
Aug 04 04:00:06.528634 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.528593    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"
Aug 04 04:00:06.663420 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.663394    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"]
Aug 04 04:00:06.665700 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:00:06.665656    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1ab1ab_a5d4_4e94_8e82_7e64923f5e48.slice/crio-5665718750c184e66030fb15ecbde18851522ee7ad6c3b5934acb4ff180e0939 WatchSource:0}: Error finding container 5665718750c184e66030fb15ecbde18851522ee7ad6c3b5934acb4ff180e0939: Status 404 returned error can't find the container with id 5665718750c184e66030fb15ecbde18851522ee7ad6c3b5934acb4ff180e0939
Aug 04 04:00:06.854049 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.853963    2559 patch_prober.go:28] interesting pod/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 04:00:06.854190 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:06.854044    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 04:00:07.126309 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:07.126206    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb" event={"ID":"ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48","Type":"ContainerStarted","Data":"fa1a346a17340a422c9d08b6d10d4cf5841c4d8a2b34510dbf13f1131f58df1c"}
Aug 04 04:00:07.126309 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:07.126260    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb" event={"ID":"ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48","Type":"ContainerStarted","Data":"5665718750c184e66030fb15ecbde18851522ee7ad6c3b5934acb4ff180e0939"}
Aug 04 04:00:07.142103 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:07.142053    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb" podStartSLOduration=1.142036633 podStartE2EDuration="1.142036633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:00:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:00:07.140718441 +0000 UTC m=+3024.438858928" watchObservedRunningTime="2026-08-04 04:00:07.142036633 +0000 UTC m=+3024.440177120"
Aug 04 04:00:08.138627 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:08.138585    2559 generic.go:356] "Generic (PLEG): container finished" podID="ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48" containerID="fa1a346a17340a422c9d08b6d10d4cf5841c4d8a2b34510dbf13f1131f58df1c" exitCode=0
Aug 04 04:00:08.139119 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:08.138636    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb" event={"ID":"ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48","Type":"ContainerDied","Data":"fa1a346a17340a422c9d08b6d10d4cf5841c4d8a2b34510dbf13f1131f58df1c"}
Aug 04 04:00:09.280144 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:09.280118    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"
Aug 04 04:00:09.324693 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:09.324648    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48-maas-api-service-ca\") pod \"ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48\" (UID: \"ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48\") "
Aug 04 04:00:09.325060 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:09.325039    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48" (UID: "ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 04:00:09.426387 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:09.426306    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48-maas-api-service-ca\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:10.149356 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:10.149318    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb" event={"ID":"ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48","Type":"ContainerDied","Data":"5665718750c184e66030fb15ecbde18851522ee7ad6c3b5934acb4ff180e0939"}
Aug 04 04:00:10.149356 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:10.149336    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"
Aug 04 04:00:10.149592 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:10.149370    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5665718750c184e66030fb15ecbde18851522ee7ad6c3b5934acb4ff180e0939"
Aug 04 04:00:11.108593 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.108561    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 04:00:11.143905 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.143824    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-data\") pod \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") "
Aug 04 04:00:11.143905 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.143886    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbdbw\" (UniqueName: \"kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-kube-api-access-vbdbw\") pod \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") "
Aug 04 04:00:11.143905 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.143904    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-envoy\") pod \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") "
Aug 04 04:00:11.144169 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.143947    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-certs\") pod \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") "
Aug 04 04:00:11.144169 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.143979    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-credential-socket\") pod \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") "
Aug 04 04:00:11.144169 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.143998    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-socket\") pod \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") "
Aug 04 04:00:11.144169 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.144045    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istiod-ca-cert\") pod \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") "
Aug 04 04:00:11.144169 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.144074    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-token\") pod \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") "
Aug 04 04:00:11.144169 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.144127    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-podinfo\") pod \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\" (UID: \"55ab5b4d-79a7-4757-bf71-6cad83cc850b\") "
Aug 04 04:00:11.144462 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.144280    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "55ab5b4d-79a7-4757-bf71-6cad83cc850b" (UID: "55ab5b4d-79a7-4757-bf71-6cad83cc850b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:00:11.144462 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.144303    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "55ab5b4d-79a7-4757-bf71-6cad83cc850b" (UID: "55ab5b4d-79a7-4757-bf71-6cad83cc850b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:00:11.144774 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.144639    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "55ab5b4d-79a7-4757-bf71-6cad83cc850b" (UID: "55ab5b4d-79a7-4757-bf71-6cad83cc850b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 04:00:11.144774 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.144652    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:11.144774 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.144677    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:11.145030 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.144902    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-data" (OuterVolumeSpecName: "istio-data") pod "55ab5b4d-79a7-4757-bf71-6cad83cc850b" (UID: "55ab5b4d-79a7-4757-bf71-6cad83cc850b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:00:11.145030 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.144913    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "55ab5b4d-79a7-4757-bf71-6cad83cc850b" (UID: "55ab5b4d-79a7-4757-bf71-6cad83cc850b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:00:11.146246 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.146208    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "55ab5b4d-79a7-4757-bf71-6cad83cc850b" (UID: "55ab5b4d-79a7-4757-bf71-6cad83cc850b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:00:11.146346 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.146298    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "55ab5b4d-79a7-4757-bf71-6cad83cc850b" (UID: "55ab5b4d-79a7-4757-bf71-6cad83cc850b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 04:00:11.146486 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.146465    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-kube-api-access-vbdbw" (OuterVolumeSpecName: "kube-api-access-vbdbw") pod "55ab5b4d-79a7-4757-bf71-6cad83cc850b" (UID: "55ab5b4d-79a7-4757-bf71-6cad83cc850b"). InnerVolumeSpecName "kube-api-access-vbdbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:00:11.146581 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.146527    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-token" (OuterVolumeSpecName: "istio-token") pod "55ab5b4d-79a7-4757-bf71-6cad83cc850b" (UID: "55ab5b4d-79a7-4757-bf71-6cad83cc850b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:00:11.156930 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.156896    2559 generic.go:356] "Generic (PLEG): container finished" podID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerID="961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c" exitCode=0
Aug 04 04:00:11.157063 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.156957    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"
Aug 04 04:00:11.157063 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.156965    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" event={"ID":"55ab5b4d-79a7-4757-bf71-6cad83cc850b","Type":"ContainerDied","Data":"961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c"}
Aug 04 04:00:11.157063 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.157022    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp" event={"ID":"55ab5b4d-79a7-4757-bf71-6cad83cc850b","Type":"ContainerDied","Data":"cbaf73a75dd90977935344cf4565671f6e77aa5186aa604f611aa0cc0f311e6c"}
Aug 04 04:00:11.157063 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.157050    2559 scope.go:117] "RemoveContainer" containerID="961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c"
Aug 04 04:00:11.171773 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.171751    2559 scope.go:117] "RemoveContainer" containerID="c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a"
Aug 04 04:00:11.180158 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.180129    2559 scope.go:117] "RemoveContainer" containerID="961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c"
Aug 04 04:00:11.180501 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:00:11.180477    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c\": container with ID starting with 961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c not found: ID does not exist" containerID="961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c"
Aug 04 04:00:11.180501 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.180510    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c"} err="failed to get container status \"961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c\": rpc error: code = NotFound desc = could not find container \"961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c\": container with ID starting with 961cf5531e7b76a06c743f8f0fe1a468e35f81eb881e767fdbcb709eea01643c not found: ID does not exist"
Aug 04 04:00:11.180706 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.180532    2559 scope.go:117] "RemoveContainer" containerID="c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a"
Aug 04 04:00:11.180912 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:00:11.180885    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a\": container with ID starting with c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a not found: ID does not exist" containerID="c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a"
Aug 04 04:00:11.180991 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.180915    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a"} err="failed to get container status \"c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a\": rpc error: code = NotFound desc = could not find container \"c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a\": container with ID starting with c05a67218434bb911edc5d5a8816d00de4d533468bebf06329a97f630ecbc28a not found: ID does not exist"
Aug 04 04:00:11.182728 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.182708    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"]
Aug 04 04:00:11.185474 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.185451    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ccfdb2-openshift-default-7db78bddb4-fz7bp"]
Aug 04 04:00:11.238883 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.238848    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" path="/var/lib/kubelet/pods/55ab5b4d-79a7-4757-bf71-6cad83cc850b/volumes"
Aug 04 04:00:11.245150 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.245122    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:11.245248 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.245163    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:11.245248 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.245180    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:11.245248 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.245190    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:11.245248 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.245198    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:11.245248 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.245207    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbdbw\" (UniqueName: \"kubernetes.io/projected/55ab5b4d-79a7-4757-bf71-6cad83cc850b-kube-api-access-vbdbw\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:11.245248 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:11.245216    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55ab5b4d-79a7-4757-bf71-6cad83cc850b-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:21.667806 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:21.667771    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:21.753691 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:21.753659    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:21.832481 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:21.832443    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:21.865043 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:21.865007    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"]
Aug 04 04:00:21.867495 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:21.867462    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-523226-9wnlb"]
Aug 04 04:00:21.933523 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:21.933418    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:22.279701 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:22.279663    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:23.237391 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:23.237359    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48" path="/var/lib/kubelet/pods/ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48/volumes"
Aug 04 04:00:28.306199 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:28.306160    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:28.569221 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:28.569138    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"]
Aug 04 04:00:28.569390 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:28.569349    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerName="istio-proxy" containerID="cri-o://cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983" gracePeriod=30
Aug 04 04:00:28.656295 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:28.656247    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:28.734251 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:28.734223    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:28.821600 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:28.821482    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:29.358364 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.358328    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:29.483119 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.483088    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"]
Aug 04 04:00:29.483450 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.483437    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48" containerName="revoke-keys"
Aug 04 04:00:29.483501 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.483452    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48" containerName="revoke-keys"
Aug 04 04:00:29.483501 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.483466    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy"
Aug 04 04:00:29.483501 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.483471    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy"
Aug 04 04:00:29.483640 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.483532    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed1ab1ab-a5d4-4e94-8e82-7e64923f5e48" containerName="revoke-keys"
Aug 04 04:00:29.483640 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.483557    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy"
Aug 04 04:00:29.483640 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.483566    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy"
Aug 04 04:00:29.486701 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.486679    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.489779 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.489756    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-07c848-openshift-default-dockercfg-6kxvx\""
Aug 04 04:00:29.502679 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.502646    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"]
Aug 04 04:00:29.514498 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.514460    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:29.612735 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.612631    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-podinfo\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.612735 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.612701    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-envoy\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.612944 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.612777    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-credential-socket\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.612944 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.612793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-data\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.612944 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.612811    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-token\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.612944 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.612888    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-certs\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.613074 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.612949    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istiod-ca-cert\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.613074 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.612976    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-socket\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.613074 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.613012    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhl8m\" (UniqueName: \"kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-kube-api-access-jhl8m\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.713717 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.713679    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhl8m\" (UniqueName: \"kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-kube-api-access-jhl8m\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.713925 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.713742    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-podinfo\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.713925 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.713785    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-envoy\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.713925 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.713811    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-credential-socket\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.713925 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.713827    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-data\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.713925 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.713844    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-token\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.713925 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.713861    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-certs\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.714256 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.714085    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istiod-ca-cert\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.714256 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.714136    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-socket\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.714397 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.714367    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-data\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.714471 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.714394    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-certs\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.714471 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.714439    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-credential-socket\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.714561 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.714470    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-socket\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.714779 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.714760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istiod-ca-cert\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.716145 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.716125    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-envoy\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.716387 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.716366    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-podinfo\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.726207 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.726172    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhl8m\" (UniqueName: \"kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-kube-api-access-jhl8m\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.727062 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.727035    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-token\") pod \"e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") " pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.797788 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.797749    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:29.943995 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.943907    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"]
Aug 04 04:00:29.946705 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:00:29.946677    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca772f5_2b52_4600_9ba7_0bf3d94c528a.slice/crio-fe5adc10aa9fe30b37b62b562d195048b46b2d7483e789aa8d9143b99128ceda WatchSource:0}: Error finding container fe5adc10aa9fe30b37b62b562d195048b46b2d7483e789aa8d9143b99128ceda: Status 404 returned error can't find the container with id fe5adc10aa9fe30b37b62b562d195048b46b2d7483e789aa8d9143b99128ceda
Aug 04 04:00:29.947964 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.947939    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:29.948993 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.948961    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:00:29.949083 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.949026    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:00:29.949083 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:29.949055    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:00:30.254508 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:30.254472    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" event={"ID":"eca772f5-2b52-4600-9ba7-0bf3d94c528a","Type":"ContainerStarted","Data":"65c6813035e7c2854fe7623eea21e42c81c1d2c5e416018e6375dcf8c9731afb"}
Aug 04 04:00:30.254716 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:30.254525    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" event={"ID":"eca772f5-2b52-4600-9ba7-0bf3d94c528a","Type":"ContainerStarted","Data":"fe5adc10aa9fe30b37b62b562d195048b46b2d7483e789aa8d9143b99128ceda"}
Aug 04 04:00:30.798188 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:30.798151    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:31.803393 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:31.803363    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:31.833464 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:31.833389    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" podStartSLOduration=2.833365395 podStartE2EDuration="2.833365395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:00:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:00:30.306525306 +0000 UTC m=+3047.604665820" watchObservedRunningTime="2026-08-04 04:00:31.833365395 +0000 UTC m=+3049.131505884"
Aug 04 04:00:32.265601 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:32.265574    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:32.266454 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:32.266433    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:32.419513 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:32.419454    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:32.548527 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:32.548448    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:33.574153 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.574113    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:33.790407 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.790377    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:33.820578 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.820529    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 04:00:33.953752 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.953661    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-envoy\") pod \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") "
Aug 04 04:00:33.953752 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.953699    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-credential-socket\") pod \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") "
Aug 04 04:00:33.953752 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.953727    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22vw4\" (UniqueName: \"kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-kube-api-access-22vw4\") pod \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") "
Aug 04 04:00:33.953752 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.953745    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-socket\") pod \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") "
Aug 04 04:00:33.954082 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.953777    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-data\") pod \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") "
Aug 04 04:00:33.954082 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.953864    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-certs\") pod \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") "
Aug 04 04:00:33.954082 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.953904    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istiod-ca-cert\") pod \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") "
Aug 04 04:00:33.954082 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.953931    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-podinfo\") pod \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") "
Aug 04 04:00:33.954082 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.953960    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-token\") pod \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\" (UID: \"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7\") "
Aug 04 04:00:33.954082 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.953989    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" (UID: "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:00:33.954698 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.954251    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:33.954698 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.954260    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" (UID: "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 04:00:33.954698 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.954430    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" (UID: "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:00:33.954865 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.954839    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" (UID: "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:00:33.954970 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.954939    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-data" (OuterVolumeSpecName: "istio-data") pod "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" (UID: "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:00:33.956282 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.956238    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" (UID: "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:00:33.956390 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.956354    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-kube-api-access-22vw4" (OuterVolumeSpecName: "kube-api-access-22vw4") pod "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" (UID: "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7"). InnerVolumeSpecName "kube-api-access-22vw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:00:33.956518 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.956495    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" (UID: "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 04:00:33.956585 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:33.956502    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-token" (OuterVolumeSpecName: "istio-token") pod "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" (UID: "1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:00:34.055458 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.055422    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:34.055458 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.055451    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22vw4\" (UniqueName: \"kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-kube-api-access-22vw4\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:34.055458 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.055461    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:34.055730 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.055472    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:34.055730 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.055480    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:34.055730 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.055487    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:34.055730 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.055498    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:34.055730 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.055507    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:00:34.176074 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.176042    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:34.272058 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.272019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:34.282513 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.276497    2559 generic.go:356] "Generic (PLEG): container finished" podID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerID="cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983" exitCode=0
Aug 04 04:00:34.282513 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.277349    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"
Aug 04 04:00:34.282513 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.277689    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" event={"ID":"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7","Type":"ContainerDied","Data":"cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983"}
Aug 04 04:00:34.282513 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.277790    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s" event={"ID":"1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7","Type":"ContainerDied","Data":"19820ad0f44a5247622ec60b559e54f3c58159070bee17d0c93dea223fd615e8"}
Aug 04 04:00:34.282513 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.277834    2559 scope.go:117] "RemoveContainer" containerID="cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983"
Aug 04 04:00:34.292612 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.292590    2559 scope.go:117] "RemoveContainer" containerID="6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1"
Aug 04 04:00:34.301356 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.301337    2559 scope.go:117] "RemoveContainer" containerID="cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983"
Aug 04 04:00:34.301805 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:00:34.301780    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983\": container with ID starting with cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983 not found: ID does not exist" containerID="cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983"
Aug 04 04:00:34.301905 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.301814    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983"} err="failed to get container status \"cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983\": rpc error: code = NotFound desc = could not find container \"cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983\": container with ID starting with cc89cb8d6ca5263bd30453919249d591893a0ec6d9949f5f8a1d432501bbc983 not found: ID does not exist"
Aug 04 04:00:34.301905 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.301832    2559 scope.go:117] "RemoveContainer" containerID="6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1"
Aug 04 04:00:34.302159 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:00:34.302139    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1\": container with ID starting with 6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1 not found: ID does not exist" containerID="6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1"
Aug 04 04:00:34.302215 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.302164    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1"} err="failed to get container status \"6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1\": rpc error: code = NotFound desc = could not find container \"6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1\": container with ID starting with 6ffc512ab400edf98f80f15c823542c3d3c4996a119ae3b6ea8f5891fce1b9e1 not found: ID does not exist"
Aug 04 04:00:34.309630 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.309602    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"]
Aug 04 04:00:34.309943 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.309932    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy"
Aug 04 04:00:34.309986 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.309945    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ab5b4d-79a7-4757-bf71-6cad83cc850b" containerName="istio-proxy"
Aug 04 04:00:34.309986 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.309957    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerName="istio-proxy"
Aug 04 04:00:34.309986 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.309962    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerName="istio-proxy"
Aug 04 04:00:34.309986 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.309971    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerName="istio-proxy"
Aug 04 04:00:34.309986 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.309976    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerName="istio-proxy"
Aug 04 04:00:34.310137 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.310036    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerName="istio-proxy"
Aug 04 04:00:34.310137 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.310045    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" containerName="istio-proxy"
Aug 04 04:00:34.314392 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.314374    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:00:34.315192 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.315173    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"]
Aug 04 04:00:34.317205 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.317182    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-07c848-dockercfg-jv7dk\""
Aug 04 04:00:34.317309 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.317211    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-07c848\""
Aug 04 04:00:34.323618 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.323593    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-523226-openshift-default-5dd479cbd6-zdx9s"]
Aug 04 04:00:34.325349 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.325327    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"]
Aug 04 04:00:34.458983 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.458944    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkmcw\" (UniqueName: \"kubernetes.io/projected/ed8d2b27-4b40-4a9e-9854-e6116715c53d-kube-api-access-vkmcw\") pod \"payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2\" (UID: \"ed8d2b27-4b40-4a9e-9854-e6116715c53d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:00:34.458983 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.458987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8d2b27-4b40-4a9e-9854-e6116715c53d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2\" (UID: \"ed8d2b27-4b40-4a9e-9854-e6116715c53d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:00:34.560225 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.560137    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkmcw\" (UniqueName: \"kubernetes.io/projected/ed8d2b27-4b40-4a9e-9854-e6116715c53d-kube-api-access-vkmcw\") pod \"payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2\" (UID: \"ed8d2b27-4b40-4a9e-9854-e6116715c53d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:00:34.560225 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.560179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8d2b27-4b40-4a9e-9854-e6116715c53d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2\" (UID: \"ed8d2b27-4b40-4a9e-9854-e6116715c53d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:00:34.560866 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.560847    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8d2b27-4b40-4a9e-9854-e6116715c53d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2\" (UID: \"ed8d2b27-4b40-4a9e-9854-e6116715c53d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:00:34.570351 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.570327    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkmcw\" (UniqueName: \"kubernetes.io/projected/ed8d2b27-4b40-4a9e-9854-e6116715c53d-kube-api-access-vkmcw\") pod \"payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2\" (UID: \"ed8d2b27-4b40-4a9e-9854-e6116715c53d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:00:34.625432 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.625356    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:00:34.778393 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:34.778330    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"]
Aug 04 04:00:34.780571 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:00:34.780531    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8d2b27_4b40_4a9e_9854_e6116715c53d.slice/crio-b28c4b81793d602c49a64e641d74c5aebb6e37a8732504e19ad5f073e098d8d2 WatchSource:0}: Error finding container b28c4b81793d602c49a64e641d74c5aebb6e37a8732504e19ad5f073e098d8d2: Status 404 returned error can't find the container with id b28c4b81793d602c49a64e641d74c5aebb6e37a8732504e19ad5f073e098d8d2
Aug 04 04:00:35.238481 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:35.238442    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7" path="/var/lib/kubelet/pods/1c58890a-a3b3-4ea8-8cd7-b4b08b23e3b7/volumes"
Aug 04 04:00:35.283317 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:35.283273    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2" event={"ID":"ed8d2b27-4b40-4a9e-9854-e6116715c53d","Type":"ContainerStarted","Data":"9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192"}
Aug 04 04:00:35.283497 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:35.283336    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2" event={"ID":"ed8d2b27-4b40-4a9e-9854-e6116715c53d","Type":"ContainerStarted","Data":"b28c4b81793d602c49a64e641d74c5aebb6e37a8732504e19ad5f073e098d8d2"}
Aug 04 04:00:35.283497 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:35.283366    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:00:35.302402 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:35.302352    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2" podStartSLOduration=1.302333007 podStartE2EDuration="1.302333007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:00:35.302073729 +0000 UTC m=+3052.600214216" watchObservedRunningTime="2026-08-04 04:00:35.302333007 +0000 UTC m=+3052.600473497"
Aug 04 04:00:46.291433 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:46.291401    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:00:47.267167 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:47.267131    2559 patch_prober.go:28] interesting pod/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Aug 04 04:00:47.267432 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:47.267233    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Aug 04 04:00:47.347931 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:47.347897    2559 generic.go:356] "Generic (PLEG): container finished" podID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerID="65c6813035e7c2854fe7623eea21e42c81c1d2c5e416018e6375dcf8c9731afb" exitCode=0
Aug 04 04:00:47.348320 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:47.347976    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" event={"ID":"eca772f5-2b52-4600-9ba7-0bf3d94c528a","Type":"ContainerDied","Data":"65c6813035e7c2854fe7623eea21e42c81c1d2c5e416018e6375dcf8c9731afb"}
Aug 04 04:00:47.348406 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:47.348392    2559 scope.go:117] "RemoveContainer" containerID="65c6813035e7c2854fe7623eea21e42c81c1d2c5e416018e6375dcf8c9731afb"
Aug 04 04:00:47.349424 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:47.349391    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:00:47.349523 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:47.349470    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:00:47.349523 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:47.349508    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:00:48.356200 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:48.356159    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" event={"ID":"eca772f5-2b52-4600-9ba7-0bf3d94c528a","Type":"ContainerStarted","Data":"dfdd53fb0207683d3039fa57b81a1457bb55fbf67eae7be1d7466d4d9b0e8f87"}
Aug 04 04:00:48.798200 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:48.798166    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:48.798405 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:48.798387    2559 patch_prober.go:28] interesting pod/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Aug 04 04:00:48.798455 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:48.798429    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Aug 04 04:00:49.798646 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:49.798605    2559 patch_prober.go:28] interesting pod/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Aug 04 04:00:49.799096 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:49.798664    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Aug 04 04:00:50.798765 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:50.798727    2559 patch_prober.go:28] interesting pod/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Aug 04 04:00:50.799189 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:50.798784    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Aug 04 04:00:51.798658 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:51.798617    2559 patch_prober.go:28] interesting pod/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Aug 04 04:00:51.799039 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:51.798683    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Aug 04 04:00:52.802107 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:52.802077    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:52.802722 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:52.802434    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:52.808994 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:52.808956    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:00:57.283612 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.283514    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:57.620704 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.620616    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:57.943339 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.943243    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"]
Aug 04 04:00:57.946510 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.946491    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:57.949190 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.949166    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-881c6f-openshift-default-dockercfg-xf25h\""
Aug 04 04:00:57.976648 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.976609    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-socket\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:57.976856 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.976666    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-credential-socket\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:57.976856 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.976738    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-certs\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:57.976856 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.976793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-envoy\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:57.976856 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.976826    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-data\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:57.976856 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.976845    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-podinfo\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:57.977126 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.976863    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1574b2-ce39-4e5f-bcee-9111a938a844-istiod-ca-cert\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:57.977126 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.976886    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sgzx\" (UniqueName: \"kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-kube-api-access-8sgzx\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:57.977126 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.977034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-token\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:57.984674 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:57.980757    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"]
Aug 04 04:00:58.014357 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.014322    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:58.077803 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.077761    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-envoy\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.077803 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.077807    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-data\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078099 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.077825    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-podinfo\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078099 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.077842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1574b2-ce39-4e5f-bcee-9111a938a844-istiod-ca-cert\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078099 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.077859    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sgzx\" (UniqueName: \"kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-kube-api-access-8sgzx\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078099 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.077899    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-token\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078099 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.077965    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-socket\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078099 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.077996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-credential-socket\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078099 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.078030    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-certs\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078463 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.078259    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-data\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078463 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.078375    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-certs\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078463 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.078426    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-socket\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078665 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.078595    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-credential-socket\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.078722 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.078698    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1574b2-ce39-4e5f-bcee-9111a938a844-istiod-ca-cert\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.080228 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.080201    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-envoy\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.080331 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.080315    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-podinfo\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.092269 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.092235    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-token\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.092386 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.092336    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sgzx\" (UniqueName: \"kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-kube-api-access-8sgzx\") pod \"e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") " pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.257226 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.257191    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:00:58.407783 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:00:58.407751    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1574b2_ce39_4e5f_bcee_9111a938a844.slice/crio-c8b64f481cb93cd6c06a66cdd4ab1446f0b1dc7c92c6bf8b752a6c4e51d54748 WatchSource:0}: Error finding container c8b64f481cb93cd6c06a66cdd4ab1446f0b1dc7c92c6bf8b752a6c4e51d54748: Status 404 returned error can't find the container with id c8b64f481cb93cd6c06a66cdd4ab1446f0b1dc7c92c6bf8b752a6c4e51d54748
Aug 04 04:00:58.408107 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.407820    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"]
Aug 04 04:00:58.409764 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.409731    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:00:58.409868 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.409811    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:00:58.409868 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.409854    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:00:58.814218 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:58.814188    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:00:59.416623 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:59.416587    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" event={"ID":"6a1574b2-ce39-4e5f-bcee-9111a938a844","Type":"ContainerStarted","Data":"4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530"}
Aug 04 04:00:59.417025 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:00:59.416653    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" event={"ID":"6a1574b2-ce39-4e5f-bcee-9111a938a844","Type":"ContainerStarted","Data":"c8b64f481cb93cd6c06a66cdd4ab1446f0b1dc7c92c6bf8b752a6c4e51d54748"}
Aug 04 04:01:00.257370 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:00.257323    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:01:00.262728 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:00.262696    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:01:00.291946 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:00.291896    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" podStartSLOduration=3.291881081 podStartE2EDuration="3.291881081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:00:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:00:59.45070459 +0000 UTC m=+3076.748845078" watchObservedRunningTime="2026-08-04 04:01:00.291881081 +0000 UTC m=+3077.590021571"
Aug 04 04:01:00.421721 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:00.421694    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:01:00.422651 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:00.422629    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:01:01.531109 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:01.531076    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:01:02.180411 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:02.180366    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:01:02.740300 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:02.740266    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:01:02.862598 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:02.862566    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:01:03.583344 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:03.583310    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:01:03.726108 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:03.726078    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:01:16.463771 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:16.463721    2559 patch_prober.go:28] interesting pod/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Aug 04 04:01:16.464270 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:16.463834    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Aug 04 04:01:17.511409 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:17.511372    2559 generic.go:356] "Generic (PLEG): container finished" podID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerID="4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530" exitCode=0
Aug 04 04:01:17.512016 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:17.511450    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" event={"ID":"6a1574b2-ce39-4e5f-bcee-9111a938a844","Type":"ContainerDied","Data":"4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530"}
Aug 04 04:01:17.512016 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:17.511996    2559 scope.go:117] "RemoveContainer" containerID="4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530"
Aug 04 04:01:17.513131 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:17.513100    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:01:17.513234 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:17.513173    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:01:17.513234 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:17.513206    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:01:18.518625 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:18.518589    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" event={"ID":"6a1574b2-ce39-4e5f-bcee-9111a938a844","Type":"ContainerStarted","Data":"0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb"}
Aug 04 04:01:19.258442 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:19.258399    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:01:19.258771 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:19.258743    2559 patch_prober.go:28] interesting pod/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Aug 04 04:01:19.258828 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:19.258798    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Aug 04 04:01:20.258619 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:20.258578    2559 patch_prober.go:28] interesting pod/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Aug 04 04:01:20.259033 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:20.258652    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Aug 04 04:01:21.258143 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:21.258105    2559 patch_prober.go:28] interesting pod/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Aug 04 04:01:21.258345 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:21.258168    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Aug 04 04:01:22.258072 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:22.258031    2559 patch_prober.go:28] interesting pod/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Aug 04 04:01:22.258479 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:22.258104    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Aug 04 04:01:23.262062 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:23.262027    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:01:23.262444 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:23.262315    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:01:23.263036 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:23.263018    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:01:26.394597 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:26.394563    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:01:31.732715 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.732680    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"]
Aug 04 04:01:31.736332 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.736306    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:31.739594 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.739536    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-07c848\"/\"kube-root-ca.crt\""
Aug 04 04:01:31.740287 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.740266    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-07c848\"/\"auth-test-model-07c848-kserve-self-signed-certs\""
Aug 04 04:01:31.740360 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.740271    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-07c848\"/\"openshift-service-ca.crt\""
Aug 04 04:01:31.740360 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.740350    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-07c848\"/\"default-dockercfg-9t4ck\""
Aug 04 04:01:31.753500 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.753465    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"]
Aug 04 04:01:31.754446 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.754413    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:01:31.908637 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.908598    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tmp-dir\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:31.908844 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.908651    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-model-cache\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:31.908844 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.908688    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tls-certs\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:31.908844 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.908827    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4qgt\" (UniqueName: \"kubernetes.io/projected/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-kube-api-access-b4qgt\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:31.909029 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.908875    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-home\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:31.909029 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:31.908902    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-dshm\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.009425 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.009330    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tmp-dir\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.009425 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.009373    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-model-cache\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.009425 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.009393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tls-certs\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.009766 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.009446    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4qgt\" (UniqueName: \"kubernetes.io/projected/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-kube-api-access-b4qgt\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.009766 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.009473    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-home\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.009766 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.009495    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-dshm\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.009937 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.009854    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-model-cache\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.009937 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.009905    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-home\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.009937 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.009932    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tmp-dir\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.011810 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.011786    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-dshm\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.012072 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.012054    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tls-certs\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.019816 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.019783    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4qgt\" (UniqueName: \"kubernetes.io/projected/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-kube-api-access-b4qgt\") pod \"auth-test-model-07c848-kserve-69d47fd859-v2c2z\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") " pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.046486 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.046441    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:32.218739 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.218710    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"]
Aug 04 04:01:32.220458 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:01:32.220431    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55a070c7_8e45_4a59_8e2a_bf0cb4ca5dde.slice/crio-d86ed0ffc3302698aa6f5b32acf8922328898e4ae96ce35bfbf6b66c8e908659 WatchSource:0}: Error finding container d86ed0ffc3302698aa6f5b32acf8922328898e4ae96ce35bfbf6b66c8e908659: Status 404 returned error can't find the container with id d86ed0ffc3302698aa6f5b32acf8922328898e4ae96ce35bfbf6b66c8e908659
Aug 04 04:01:32.598185 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.598092    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z" event={"ID":"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde","Type":"ContainerStarted","Data":"de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5"}
Aug 04 04:01:32.598185 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:32.598133    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z" event={"ID":"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde","Type":"ContainerStarted","Data":"d86ed0ffc3302698aa6f5b32acf8922328898e4ae96ce35bfbf6b66c8e908659"}
Aug 04 04:01:42.047631 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:42.047584    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:42.047631 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:42.047644    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:42.060375 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:42.060343    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:42.081517 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:42.081457    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z" podStartSLOduration=11.081440084 podStartE2EDuration="11.081440084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:01:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:01:32.644995431 +0000 UTC m=+3109.943135920" watchObservedRunningTime="2026-08-04 04:01:42.081440084 +0000 UTC m=+3119.379580650"
Aug 04 04:01:42.664141 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:42.664110    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:01:43.778579 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:01:43.778532    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:02:02.098287 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.098248    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"]
Aug 04 04:02:02.102068 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.102040    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.105857 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.105832    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"default-dockercfg-frzsc\""
Aug 04 04:02:02.106040 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.105833    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"kube-root-ca.crt\""
Aug 04 04:02:02.106040 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.105833    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"auth-test-model-881c6f-kserve-self-signed-certs\""
Aug 04 04:02:02.106040 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.105869    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"openshift-service-ca.crt\""
Aug 04 04:02:02.115700 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.115666    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"]
Aug 04 04:02:02.138317 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.138282    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:02:02.191866 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.191819    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-dshm\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.192052 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.191997    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tls-certs\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.192095 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.192061    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tmp-dir\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.192137 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.192102    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-model-cache\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.192172 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.192147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-home\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.192207 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.192176    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkn6v\" (UniqueName: \"kubernetes.io/projected/c6089648-fecd-4a4b-b8ea-9b385f354e3a-kube-api-access-qkn6v\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.293438 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.293394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tls-certs\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.293684 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.293454    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tmp-dir\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.293684 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.293496    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-model-cache\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.293684 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.293521    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-home\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.293684 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.293582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkn6v\" (UniqueName: \"kubernetes.io/projected/c6089648-fecd-4a4b-b8ea-9b385f354e3a-kube-api-access-qkn6v\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.293684 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.293613    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-dshm\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.294068 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.294035    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tmp-dir\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.294370 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.294344    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-home\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.294644 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.294619    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-model-cache\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.297013 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.296978    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-dshm\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.297778 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.297748    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tls-certs\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.306154 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.306110    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkn6v\" (UniqueName: \"kubernetes.io/projected/c6089648-fecd-4a4b-b8ea-9b385f354e3a-kube-api-access-qkn6v\") pod \"auth-test-model-881c6f-kserve-8644b8d655-7m8s6\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") " pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.413592 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.413461    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:02.574573 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.574507    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"]
Aug 04 04:02:02.576006 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:02:02.575981    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6089648_fecd_4a4b_b8ea_9b385f354e3a.slice/crio-123624ce5632bf4822a3a42434a21594810f5fe5be6a629da7facd71a5958868 WatchSource:0}: Error finding container 123624ce5632bf4822a3a42434a21594810f5fe5be6a629da7facd71a5958868: Status 404 returned error can't find the container with id 123624ce5632bf4822a3a42434a21594810f5fe5be6a629da7facd71a5958868
Aug 04 04:02:02.765338 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.765287    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6" event={"ID":"c6089648-fecd-4a4b-b8ea-9b385f354e3a","Type":"ContainerStarted","Data":"0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547"}
Aug 04 04:02:02.765338 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.765344    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6" event={"ID":"c6089648-fecd-4a4b-b8ea-9b385f354e3a","Type":"ContainerStarted","Data":"123624ce5632bf4822a3a42434a21594810f5fe5be6a629da7facd71a5958868"}
Aug 04 04:02:02.788294 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:02.788231    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6" podStartSLOduration=0.788182997 podStartE2EDuration="788.182997ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:02:02.787438697 +0000 UTC m=+3140.085579185" watchObservedRunningTime="2026-08-04 04:02:02.788182997 +0000 UTC m=+3140.086323486"
Aug 04 04:02:12.413768 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:12.413656    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:12.413768 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:12.413729    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:12.426674 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:12.426646    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:02:12.832334 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:02:12.832302    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:04:18.084320 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:04:18.084235    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:04:43.645153 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:04:43.645116    2559 scope.go:117] "RemoveContainer" containerID="8499d47b027fc68485d6e92f702817d13b7c569b2a9c539f117ae417431fd2cc"
Aug 04 04:05:46.747795 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:05:46.747752    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:06:43.673210 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:06:43.673129    2559 scope.go:117] "RemoveContainer" containerID="fa1a346a17340a422c9d08b6d10d4cf5841c4d8a2b34510dbf13f1131f58df1c"
Aug 04 04:06:43.682096 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:06:43.682076    2559 scope.go:117] "RemoveContainer" containerID="957b6416ea2eb3ec44b247c10f7bf5a96f5d18dcced97c28230b24cb22c5823f"
Aug 04 04:06:51.998280 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:06:51.998224    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:06:56.772437 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:06:56.772401    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"]
Aug 04 04:06:56.801246 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:06:56.801199    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:06:57.819602 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:06:57.819562    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"]
Aug 04 04:06:57.841703 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:06:57.841675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:07:09.324973 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:09.324939    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:07:11.791315 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:11.791266    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z" podUID="55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" containerName="main" containerID="cri-o://de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5" gracePeriod=45
Aug 04 04:07:12.044379 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.044312    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:07:12.101012 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.100977    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tmp-dir\") pod \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") "
Aug 04 04:07:12.101190 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.101025    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-model-cache\") pod \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") "
Aug 04 04:07:12.101190 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.101082    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4qgt\" (UniqueName: \"kubernetes.io/projected/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-kube-api-access-b4qgt\") pod \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") "
Aug 04 04:07:12.101190 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.101115    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-dshm\") pod \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") "
Aug 04 04:07:12.101190 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.101145    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-home\") pod \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") "
Aug 04 04:07:12.101365 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.101237    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tls-certs\") pod \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\" (UID: \"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde\") "
Aug 04 04:07:12.101365 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.101262    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" (UID: "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:07:12.101365 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.101342    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-model-cache" (OuterVolumeSpecName: "model-cache") pod "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" (UID: "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:07:12.101496 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.101455    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-home" (OuterVolumeSpecName: "home") pod "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" (UID: "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:07:12.101586 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.101567    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-model-cache\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:12.101646 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.101592    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-home\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:12.101646 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.101605    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tmp-dir\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:12.103328 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.103302    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-dshm" (OuterVolumeSpecName: "dshm") pod "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" (UID: "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:07:12.103328 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.103305    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" (UID: "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 04:07:12.103479 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.103359    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-kube-api-access-b4qgt" (OuterVolumeSpecName: "kube-api-access-b4qgt") pod "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" (UID: "55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde"). InnerVolumeSpecName "kube-api-access-b4qgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:07:12.202296 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.202256    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-dshm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:12.202296 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.202290    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-tls-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:12.202296 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.202301    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4qgt\" (UniqueName: \"kubernetes.io/projected/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde-kube-api-access-b4qgt\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:12.396533 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.396438    2559 generic.go:356] "Generic (PLEG): container finished" podID="55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" containerID="de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5" exitCode=0
Aug 04 04:07:12.396533 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.396503    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"
Aug 04 04:07:12.396533 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.396533    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z" event={"ID":"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde","Type":"ContainerDied","Data":"de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5"}
Aug 04 04:07:12.396830 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.396598    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z" event={"ID":"55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde","Type":"ContainerDied","Data":"d86ed0ffc3302698aa6f5b32acf8922328898e4ae96ce35bfbf6b66c8e908659"}
Aug 04 04:07:12.396830 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.396620    2559 scope.go:117] "RemoveContainer" containerID="de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5"
Aug 04 04:07:12.405608 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.405589    2559 scope.go:117] "RemoveContainer" containerID="de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5"
Aug 04 04:07:12.405884 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:07:12.405865    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5\": container with ID starting with de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5 not found: ID does not exist" containerID="de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5"
Aug 04 04:07:12.405947 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.405897    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5"} err="failed to get container status \"de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5\": rpc error: code = NotFound desc = could not find container \"de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5\": container with ID starting with de5d6c4df31bba60ac3c7b54957c000abb826586750215169460ce7435fda0a5 not found: ID does not exist"
Aug 04 04:07:12.417745 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.417715    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"]
Aug 04 04:07:12.421699 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.421673    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-07c848/auth-test-model-07c848-kserve-69d47fd859-v2c2z"]
Aug 04 04:07:12.840455 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:12.840412    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6" podUID="c6089648-fecd-4a4b-b8ea-9b385f354e3a" containerName="main" containerID="cri-o://0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547" gracePeriod=45
Aug 04 04:07:13.103062 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.102988    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:07:13.211321 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211284    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-dshm\") pod \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") "
Aug 04 04:07:13.211321 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211322    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tls-certs\") pod \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") "
Aug 04 04:07:13.211595 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211361    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tmp-dir\") pod \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") "
Aug 04 04:07:13.211595 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211379    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-model-cache\") pod \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") "
Aug 04 04:07:13.211595 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211428    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-home\") pod \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") "
Aug 04 04:07:13.211595 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211450    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkn6v\" (UniqueName: \"kubernetes.io/projected/c6089648-fecd-4a4b-b8ea-9b385f354e3a-kube-api-access-qkn6v\") pod \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\" (UID: \"c6089648-fecd-4a4b-b8ea-9b385f354e3a\") "
Aug 04 04:07:13.211810 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211658    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c6089648-fecd-4a4b-b8ea-9b385f354e3a" (UID: "c6089648-fecd-4a4b-b8ea-9b385f354e3a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:07:13.211810 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211736    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-model-cache" (OuterVolumeSpecName: "model-cache") pod "c6089648-fecd-4a4b-b8ea-9b385f354e3a" (UID: "c6089648-fecd-4a4b-b8ea-9b385f354e3a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:07:13.211810 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211749    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-home" (OuterVolumeSpecName: "home") pod "c6089648-fecd-4a4b-b8ea-9b385f354e3a" (UID: "c6089648-fecd-4a4b-b8ea-9b385f354e3a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:07:13.211954 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211860    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tmp-dir\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:13.211954 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211879    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-model-cache\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:13.211954 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.211894    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-home\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:13.213667 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.213638    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6089648-fecd-4a4b-b8ea-9b385f354e3a-kube-api-access-qkn6v" (OuterVolumeSpecName: "kube-api-access-qkn6v") pod "c6089648-fecd-4a4b-b8ea-9b385f354e3a" (UID: "c6089648-fecd-4a4b-b8ea-9b385f354e3a"). InnerVolumeSpecName "kube-api-access-qkn6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:07:13.213768 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.213680    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c6089648-fecd-4a4b-b8ea-9b385f354e3a" (UID: "c6089648-fecd-4a4b-b8ea-9b385f354e3a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 04:07:13.213768 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.213710    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-dshm" (OuterVolumeSpecName: "dshm") pod "c6089648-fecd-4a4b-b8ea-9b385f354e3a" (UID: "c6089648-fecd-4a4b-b8ea-9b385f354e3a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:07:13.242646 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.242606    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" path="/var/lib/kubelet/pods/55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde/volumes"
Aug 04 04:07:13.313103 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.313062    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6089648-fecd-4a4b-b8ea-9b385f354e3a-dshm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:13.313274 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.313120    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6089648-fecd-4a4b-b8ea-9b385f354e3a-tls-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:13.313274 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.313131    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkn6v\" (UniqueName: \"kubernetes.io/projected/c6089648-fecd-4a4b-b8ea-9b385f354e3a-kube-api-access-qkn6v\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:07:13.403932 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.403848    2559 generic.go:356] "Generic (PLEG): container finished" podID="c6089648-fecd-4a4b-b8ea-9b385f354e3a" containerID="0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547" exitCode=0
Aug 04 04:07:13.403932 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.403912    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"
Aug 04 04:07:13.404095 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.403928    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6" event={"ID":"c6089648-fecd-4a4b-b8ea-9b385f354e3a","Type":"ContainerDied","Data":"0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547"}
Aug 04 04:07:13.404095 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.403977    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6" event={"ID":"c6089648-fecd-4a4b-b8ea-9b385f354e3a","Type":"ContainerDied","Data":"123624ce5632bf4822a3a42434a21594810f5fe5be6a629da7facd71a5958868"}
Aug 04 04:07:13.404095 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.403999    2559 scope.go:117] "RemoveContainer" containerID="0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547"
Aug 04 04:07:13.412635 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.412616    2559 scope.go:117] "RemoveContainer" containerID="0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547"
Aug 04 04:07:13.412953 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:07:13.412929    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547\": container with ID starting with 0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547 not found: ID does not exist" containerID="0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547"
Aug 04 04:07:13.413051 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.412960    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547"} err="failed to get container status \"0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547\": rpc error: code = NotFound desc = could not find container \"0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547\": container with ID starting with 0f0f2c41a1977f4d37a558da7b8e4f021cd979989963ceb630370b2245b43547 not found: ID does not exist"
Aug 04 04:07:13.421577 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.421526    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"]
Aug 04 04:07:13.429683 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:13.429650    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/auth-test-model-881c6f-kserve-8644b8d655-7m8s6"]
Aug 04 04:07:15.237603 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:15.237567    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6089648-fecd-4a4b-b8ea-9b385f354e3a" path="/var/lib/kubelet/pods/c6089648-fecd-4a4b-b8ea-9b385f354e3a/volumes"
Aug 04 04:07:41.396416 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.396376    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"]
Aug 04 04:07:41.396836 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.396799    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" containerName="main"
Aug 04 04:07:41.396836 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.396812    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" containerName="main"
Aug 04 04:07:41.396836 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.396825    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6089648-fecd-4a4b-b8ea-9b385f354e3a" containerName="main"
Aug 04 04:07:41.396836 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.396831    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6089648-fecd-4a4b-b8ea-9b385f354e3a" containerName="main"
Aug 04 04:07:41.396979 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.396912    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="55a070c7-8e45-4a59-8e2a-bf0cb4ca5dde" containerName="main"
Aug 04 04:07:41.396979 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.396922    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6089648-fecd-4a4b-b8ea-9b385f354e3a" containerName="main"
Aug 04 04:07:41.399889 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.399864    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.403353 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.403330    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"kube-root-ca.crt\""
Aug 04 04:07:41.403572 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.403535    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"default-dockercfg-frzsc\""
Aug 04 04:07:41.404231 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.404215    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"sub-test-model-881c6f-kserve-self-signed-certs\""
Aug 04 04:07:41.407925 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.407904    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"openshift-service-ca.crt\""
Aug 04 04:07:41.424812 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.424773    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"]
Aug 04 04:07:41.458518 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.458482    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:07:41.471839 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.471794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-dshm\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.472050 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.471852    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tmp-dir\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.472050 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.471878    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tls-certs\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.472050 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.471921    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-home\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.472050 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.471951    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-model-cache\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.472050 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.471981    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfz9x\" (UniqueName: \"kubernetes.io/projected/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-kube-api-access-sfz9x\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.572918 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.572877    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-home\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.572918 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.572924    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-model-cache\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.573190 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.572950    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfz9x\" (UniqueName: \"kubernetes.io/projected/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-kube-api-access-sfz9x\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.573190 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.573018    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-dshm\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.573190 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.573045    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tmp-dir\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.573190 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.573060    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tls-certs\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.573412 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.573363    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-home\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.573412 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.573385    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-model-cache\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.573485 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.573470    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tmp-dir\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.575407 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.575387    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-dshm\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.575667 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.575650    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tls-certs\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.584001 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.583967    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfz9x\" (UniqueName: \"kubernetes.io/projected/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-kube-api-access-sfz9x\") pod \"sub-test-model-881c6f-kserve-6c6b958fbc-htcph\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") " pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.709901 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.709791    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:41.862519 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:41.862483    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"]
Aug 04 04:07:41.864593 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:07:41.864559    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd79eedd9_7ee8_4510_9dc3_3781b597fb9c.slice/crio-11d11b782c4e238721efa4809f94f7e0a13bc929fe5c84e9a854103a477949a7 WatchSource:0}: Error finding container 11d11b782c4e238721efa4809f94f7e0a13bc929fe5c84e9a854103a477949a7: Status 404 returned error can't find the container with id 11d11b782c4e238721efa4809f94f7e0a13bc929fe5c84e9a854103a477949a7
Aug 04 04:07:42.552895 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:42.552863    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph" event={"ID":"d79eedd9-7ee8-4510-9dc3-3781b597fb9c","Type":"ContainerStarted","Data":"c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963"}
Aug 04 04:07:42.553289 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:42.552903    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph" event={"ID":"d79eedd9-7ee8-4510-9dc3-3781b597fb9c","Type":"ContainerStarted","Data":"11d11b782c4e238721efa4809f94f7e0a13bc929fe5c84e9a854103a477949a7"}
Aug 04 04:07:42.578251 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:42.578195    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph" podStartSLOduration=1.578179233 podStartE2EDuration="1.578179233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:07:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:07:42.576222973 +0000 UTC m=+3479.874363461" watchObservedRunningTime="2026-08-04 04:07:42.578179233 +0000 UTC m=+3479.876319721"
Aug 04 04:07:45.729485 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:45.729447    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:07:51.711019 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:51.710977    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:51.711687 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:51.711047    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:51.723775 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:51.723750    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:07:52.615055 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:07:52.615023    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:08:52.501873 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:08:52.501837    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:08:52.565948 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:08:52.565917    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:09:28.951423 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:28.951372    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:09:29.895749 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:29.895714    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"]
Aug 04 04:09:29.917150 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:29.917119    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:09:38.377959 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.377863    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"]
Aug 04 04:09:38.381511 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.381483    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.384745 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.384711    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-07c848\"/\"default-dockercfg-9t4ck\""
Aug 04 04:09:38.384891 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.384746    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-07c848\"/\"openshift-service-ca.crt\""
Aug 04 04:09:38.384891 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.384747    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-07c848\"/\"kube-root-ca.crt\""
Aug 04 04:09:38.384891 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.384812    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-07c848\"/\"rate-test-model-07c848-kserve-self-signed-certs\""
Aug 04 04:09:38.390734 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.390708    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"]
Aug 04 04:09:38.394576 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.394502    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgxgk\" (UniqueName: \"kubernetes.io/projected/e4cc7925-3779-4140-b76a-25d682b4c59f-kube-api-access-pgxgk\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.394740 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.394596    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-dshm\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.394740 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.394638    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4cc7925-3779-4140-b76a-25d682b4c59f-tls-certs\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.394740 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.394718    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-home\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.394861 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.394753    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-model-cache\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.394861 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.394849    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-tmp-dir\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.401591 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.401532    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:09:38.495990 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.495945    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-tmp-dir\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.496204 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.496012    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgxgk\" (UniqueName: \"kubernetes.io/projected/e4cc7925-3779-4140-b76a-25d682b4c59f-kube-api-access-pgxgk\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.496204 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.496046    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-dshm\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.496204 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.496068    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4cc7925-3779-4140-b76a-25d682b4c59f-tls-certs\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.496204 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.496113    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-home\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.496204 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.496151    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-model-cache\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.496511 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.496473    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-tmp-dir\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.496603 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.496510    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-home\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.496650 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.496614    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-model-cache\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.498496 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.498466    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-dshm\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.498980 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.498959    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4cc7925-3779-4140-b76a-25d682b4c59f-tls-certs\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.504764 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.504736    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgxgk\" (UniqueName: \"kubernetes.io/projected/e4cc7925-3779-4140-b76a-25d682b4c59f-kube-api-access-pgxgk\") pod \"rate-test-model-07c848-kserve-7dd76c7987-7p878\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") " pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.691582 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.691462    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:38.833781 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:38.833752    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"]
Aug 04 04:09:38.836712 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:09:38.836676    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4cc7925_3779_4140_b76a_25d682b4c59f.slice/crio-a767d30959951b5dce37b771ec212d506f6c89d2cc05900f34c1b8ef2b8f9e0c WatchSource:0}: Error finding container a767d30959951b5dce37b771ec212d506f6c89d2cc05900f34c1b8ef2b8f9e0c: Status 404 returned error can't find the container with id a767d30959951b5dce37b771ec212d506f6c89d2cc05900f34c1b8ef2b8f9e0c
Aug 04 04:09:39.155777 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:39.155732    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878" event={"ID":"e4cc7925-3779-4140-b76a-25d682b4c59f","Type":"ContainerStarted","Data":"8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02"}
Aug 04 04:09:39.155777 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:39.155773    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878" event={"ID":"e4cc7925-3779-4140-b76a-25d682b4c59f","Type":"ContainerStarted","Data":"a767d30959951b5dce37b771ec212d506f6c89d2cc05900f34c1b8ef2b8f9e0c"}
Aug 04 04:09:39.173018 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:39.172957    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878" podStartSLOduration=1.172938515 podStartE2EDuration="1.172938515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:09:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:09:39.171736366 +0000 UTC m=+3596.469876867" watchObservedRunningTime="2026-08-04 04:09:39.172938515 +0000 UTC m=+3596.471079029"
Aug 04 04:09:44.915689 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:44.915645    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph" podUID="d79eedd9-7ee8-4510-9dc3-3781b597fb9c" containerName="main" containerID="cri-o://c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963" gracePeriod=45
Aug 04 04:09:45.175803 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.175717    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:09:45.187596 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.187549    2559 generic.go:356] "Generic (PLEG): container finished" podID="d79eedd9-7ee8-4510-9dc3-3781b597fb9c" containerID="c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963" exitCode=0
Aug 04 04:09:45.187596 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.187576    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph" event={"ID":"d79eedd9-7ee8-4510-9dc3-3781b597fb9c","Type":"ContainerDied","Data":"c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963"}
Aug 04 04:09:45.187596 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.187604    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"
Aug 04 04:09:45.187950 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.187638    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph" event={"ID":"d79eedd9-7ee8-4510-9dc3-3781b597fb9c","Type":"ContainerDied","Data":"11d11b782c4e238721efa4809f94f7e0a13bc929fe5c84e9a854103a477949a7"}
Aug 04 04:09:45.187950 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.187662    2559 scope.go:117] "RemoveContainer" containerID="c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963"
Aug 04 04:09:45.206245 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.201819    2559 scope.go:117] "RemoveContainer" containerID="c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963"
Aug 04 04:09:45.206955 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:09:45.206926    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963\": container with ID starting with c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963 not found: ID does not exist" containerID="c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963"
Aug 04 04:09:45.207086 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.206965    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963"} err="failed to get container status \"c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963\": rpc error: code = NotFound desc = could not find container \"c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963\": container with ID starting with c017fd25b6f45e1aab76928aa91e69ec45aad45689448729b6d2078f8c04d963 not found: ID does not exist"
Aug 04 04:09:45.262465 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.262421    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tls-certs\") pod \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") "
Aug 04 04:09:45.262465 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.262476    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-dshm\") pod \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") "
Aug 04 04:09:45.262770 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.262505    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tmp-dir\") pod \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") "
Aug 04 04:09:45.262770 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.262522    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfz9x\" (UniqueName: \"kubernetes.io/projected/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-kube-api-access-sfz9x\") pod \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") "
Aug 04 04:09:45.262770 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.262613    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-model-cache\") pod \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") "
Aug 04 04:09:45.262770 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.262644    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-home\") pod \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\" (UID: \"d79eedd9-7ee8-4510-9dc3-3781b597fb9c\") "
Aug 04 04:09:45.262984 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.262841    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d79eedd9-7ee8-4510-9dc3-3781b597fb9c" (UID: "d79eedd9-7ee8-4510-9dc3-3781b597fb9c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:09:45.262984 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.262914    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-model-cache" (OuterVolumeSpecName: "model-cache") pod "d79eedd9-7ee8-4510-9dc3-3781b597fb9c" (UID: "d79eedd9-7ee8-4510-9dc3-3781b597fb9c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:09:45.263095 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.262997    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-home" (OuterVolumeSpecName: "home") pod "d79eedd9-7ee8-4510-9dc3-3781b597fb9c" (UID: "d79eedd9-7ee8-4510-9dc3-3781b597fb9c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:09:45.263095 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.263084    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-home\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:09:45.263195 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.263104    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tmp-dir\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:09:45.263195 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.263119    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-model-cache\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:09:45.264787 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.264758    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-dshm" (OuterVolumeSpecName: "dshm") pod "d79eedd9-7ee8-4510-9dc3-3781b597fb9c" (UID: "d79eedd9-7ee8-4510-9dc3-3781b597fb9c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:09:45.265198 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.265174    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d79eedd9-7ee8-4510-9dc3-3781b597fb9c" (UID: "d79eedd9-7ee8-4510-9dc3-3781b597fb9c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 04:09:45.265251 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.265192    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-kube-api-access-sfz9x" (OuterVolumeSpecName: "kube-api-access-sfz9x") pod "d79eedd9-7ee8-4510-9dc3-3781b597fb9c" (UID: "d79eedd9-7ee8-4510-9dc3-3781b597fb9c"). InnerVolumeSpecName "kube-api-access-sfz9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:09:45.364418 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.364372    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-tls-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:09:45.364418 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.364415    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-dshm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:09:45.364418 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.364426    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfz9x\" (UniqueName: \"kubernetes.io/projected/d79eedd9-7ee8-4510-9dc3-3781b597fb9c-kube-api-access-sfz9x\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:09:45.509114 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.509076    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"]
Aug 04 04:09:45.514189 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:45.514156    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/sub-test-model-881c6f-kserve-6c6b958fbc-htcph"]
Aug 04 04:09:47.238641 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:47.238601    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d79eedd9-7ee8-4510-9dc3-3781b597fb9c" path="/var/lib/kubelet/pods/d79eedd9-7ee8-4510-9dc3-3781b597fb9c/volumes"
Aug 04 04:09:48.692321 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:48.692265    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:48.692321 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:48.692335    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:48.705670 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:48.705635    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:49.221212 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:49.221181    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:09:59.099678 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.099643    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"]
Aug 04 04:09:59.100084 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.100059    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d79eedd9-7ee8-4510-9dc3-3781b597fb9c" containerName="main"
Aug 04 04:09:59.100084 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.100070    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d79eedd9-7ee8-4510-9dc3-3781b597fb9c" containerName="main"
Aug 04 04:09:59.100160 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.100137    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d79eedd9-7ee8-4510-9dc3-3781b597fb9c" containerName="main"
Aug 04 04:09:59.102876 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.102854    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.105181 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.105155    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"kube-root-ca.crt\""
Aug 04 04:09:59.106025 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.106004    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"openshift-service-ca.crt\""
Aug 04 04:09:59.106025 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.106020    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"default-dockercfg-frzsc\""
Aug 04 04:09:59.106189 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.106089    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-881c6f\"/\"rate-test-model-881c6f-kserve-self-signed-certs\""
Aug 04 04:09:59.112425 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.112393    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"]
Aug 04 04:09:59.129316 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.129278    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:09:59.197025 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.196967    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dede7bf-fe31-4550-8077-b1039eee0603-tls-certs\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.197025 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.197026    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nmst\" (UniqueName: \"kubernetes.io/projected/5dede7bf-fe31-4550-8077-b1039eee0603-kube-api-access-5nmst\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.197310 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.197102    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-home\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.197310 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.197142    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-dshm\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.197310 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.197208    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-model-cache\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.197310 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.197248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-tmp-dir\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.298840 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.298788    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-home\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.298840 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.298825    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-dshm\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.299111 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.298911    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-model-cache\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.299172 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.299099    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-tmp-dir\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.299269 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.299247    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dede7bf-fe31-4550-8077-b1039eee0603-tls-certs\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.299488 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.299255    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-home\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.299488 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.299479    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-tmp-dir\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.299683 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.299309    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-model-cache\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.299683 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.299467    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nmst\" (UniqueName: \"kubernetes.io/projected/5dede7bf-fe31-4550-8077-b1039eee0603-kube-api-access-5nmst\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.301428 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.301396    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-dshm\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.302499 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.302478    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dede7bf-fe31-4550-8077-b1039eee0603-tls-certs\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.307528 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.307500    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nmst\" (UniqueName: \"kubernetes.io/projected/5dede7bf-fe31-4550-8077-b1039eee0603-kube-api-access-5nmst\") pod \"rate-test-model-881c6f-kserve-8bf6996dc-w54nt\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") " pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.413077 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.412979    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:09:59.552084 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:09:59.552055    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"]
Aug 04 04:09:59.554555 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:09:59.554511    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5dede7bf_fe31_4550_8077_b1039eee0603.slice/crio-d3764998b2a36b5ad0842eea9a01a75f9bd46ed1c1661b39e43355cb88b32c9a WatchSource:0}: Error finding container d3764998b2a36b5ad0842eea9a01a75f9bd46ed1c1661b39e43355cb88b32c9a: Status 404 returned error can't find the container with id d3764998b2a36b5ad0842eea9a01a75f9bd46ed1c1661b39e43355cb88b32c9a
Aug 04 04:10:00.267450 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:10:00.267418    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt" event={"ID":"5dede7bf-fe31-4550-8077-b1039eee0603","Type":"ContainerStarted","Data":"f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc"}
Aug 04 04:10:00.267967 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:10:00.267458    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt" event={"ID":"5dede7bf-fe31-4550-8077-b1039eee0603","Type":"ContainerStarted","Data":"d3764998b2a36b5ad0842eea9a01a75f9bd46ed1c1661b39e43355cb88b32c9a"}
Aug 04 04:10:00.284913 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:10:00.284863    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt" podStartSLOduration=1.284845371 podStartE2EDuration="1.284845371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:09:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:10:00.283379725 +0000 UTC m=+3617.581520230" watchObservedRunningTime="2026-08-04 04:10:00.284845371 +0000 UTC m=+3617.582985857"
Aug 04 04:10:06.299531 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:10:06.299456    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:10:09.413144 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:10:09.413107    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:10:09.413675 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:10:09.413222    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:10:09.425560 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:10:09.425518    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:10:10.328068 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:10:10.328039    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:11:17.430420 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:11:17.430304    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:11:51.171599 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:11:51.171528    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"]
Aug 04 04:11:51.220112 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:11:51.220071    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:11:52.175584 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:11:52.175536    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"]
Aug 04 04:11:52.192933 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:11:52.192902    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:06.190871 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.190828    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878" podUID="e4cc7925-3779-4140-b76a-25d682b4c59f" containerName="main" containerID="cri-o://8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02" gracePeriod=45
Aug 04 04:12:06.444959 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.444890    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:12:06.532892 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.532856    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4cc7925-3779-4140-b76a-25d682b4c59f-tls-certs\") pod \"e4cc7925-3779-4140-b76a-25d682b4c59f\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") "
Aug 04 04:12:06.532892 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.532894    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-model-cache\") pod \"e4cc7925-3779-4140-b76a-25d682b4c59f\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") "
Aug 04 04:12:06.533163 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.532920    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgxgk\" (UniqueName: \"kubernetes.io/projected/e4cc7925-3779-4140-b76a-25d682b4c59f-kube-api-access-pgxgk\") pod \"e4cc7925-3779-4140-b76a-25d682b4c59f\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") "
Aug 04 04:12:06.533163 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.532940    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-tmp-dir\") pod \"e4cc7925-3779-4140-b76a-25d682b4c59f\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") "
Aug 04 04:12:06.533163 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.532976    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-dshm\") pod \"e4cc7925-3779-4140-b76a-25d682b4c59f\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") "
Aug 04 04:12:06.533163 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.533073    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-home\") pod \"e4cc7925-3779-4140-b76a-25d682b4c59f\" (UID: \"e4cc7925-3779-4140-b76a-25d682b4c59f\") "
Aug 04 04:12:06.533373 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.533289    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-model-cache" (OuterVolumeSpecName: "model-cache") pod "e4cc7925-3779-4140-b76a-25d682b4c59f" (UID: "e4cc7925-3779-4140-b76a-25d682b4c59f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:12:06.533373 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.533307    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-home" (OuterVolumeSpecName: "home") pod "e4cc7925-3779-4140-b76a-25d682b4c59f" (UID: "e4cc7925-3779-4140-b76a-25d682b4c59f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:12:06.533373 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.533328    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e4cc7925-3779-4140-b76a-25d682b4c59f" (UID: "e4cc7925-3779-4140-b76a-25d682b4c59f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:12:06.535355 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.535324    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-dshm" (OuterVolumeSpecName: "dshm") pod "e4cc7925-3779-4140-b76a-25d682b4c59f" (UID: "e4cc7925-3779-4140-b76a-25d682b4c59f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:12:06.535815 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.535787    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4cc7925-3779-4140-b76a-25d682b4c59f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e4cc7925-3779-4140-b76a-25d682b4c59f" (UID: "e4cc7925-3779-4140-b76a-25d682b4c59f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 04:12:06.535930 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.535795    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4cc7925-3779-4140-b76a-25d682b4c59f-kube-api-access-pgxgk" (OuterVolumeSpecName: "kube-api-access-pgxgk") pod "e4cc7925-3779-4140-b76a-25d682b4c59f" (UID: "e4cc7925-3779-4140-b76a-25d682b4c59f"). InnerVolumeSpecName "kube-api-access-pgxgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:12:06.634439 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.634403    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-dshm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:06.634439 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.634432    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-home\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:06.634439 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.634441    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4cc7925-3779-4140-b76a-25d682b4c59f-tls-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:06.634696 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.634452    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-model-cache\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:06.634696 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.634462    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgxgk\" (UniqueName: \"kubernetes.io/projected/e4cc7925-3779-4140-b76a-25d682b4c59f-kube-api-access-pgxgk\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:06.634696 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.634470    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4cc7925-3779-4140-b76a-25d682b4c59f-tmp-dir\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:06.919690 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.919652    2559 generic.go:356] "Generic (PLEG): container finished" podID="e4cc7925-3779-4140-b76a-25d682b4c59f" containerID="8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02" exitCode=0
Aug 04 04:12:06.919883 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.919706    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"
Aug 04 04:12:06.919883 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.919756    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878" event={"ID":"e4cc7925-3779-4140-b76a-25d682b4c59f","Type":"ContainerDied","Data":"8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02"}
Aug 04 04:12:06.919883 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.919799    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878" event={"ID":"e4cc7925-3779-4140-b76a-25d682b4c59f","Type":"ContainerDied","Data":"a767d30959951b5dce37b771ec212d506f6c89d2cc05900f34c1b8ef2b8f9e0c"}
Aug 04 04:12:06.919883 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.919816    2559 scope.go:117] "RemoveContainer" containerID="8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02"
Aug 04 04:12:06.929108 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.929083    2559 scope.go:117] "RemoveContainer" containerID="8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02"
Aug 04 04:12:06.929434 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:12:06.929416    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02\": container with ID starting with 8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02 not found: ID does not exist" containerID="8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02"
Aug 04 04:12:06.929488 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.929443    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02"} err="failed to get container status \"8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02\": rpc error: code = NotFound desc = could not find container \"8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02\": container with ID starting with 8a073f4305425274d38cb15d21e23fdd53480bd7fa229aaa4c7127996bb9cc02 not found: ID does not exist"
Aug 04 04:12:06.942668 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.942635    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"]
Aug 04 04:12:06.947164 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:06.947135    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-07c848/rate-test-model-07c848-kserve-7dd76c7987-7p878"]
Aug 04 04:12:07.194839 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.194729    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt" podUID="5dede7bf-fe31-4550-8077-b1039eee0603" containerName="main" containerID="cri-o://f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc" gracePeriod=45
Aug 04 04:12:07.244953 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.241823    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4cc7925-3779-4140-b76a-25d682b4c59f" path="/var/lib/kubelet/pods/e4cc7925-3779-4140-b76a-25d682b4c59f/volumes"
Aug 04 04:12:07.452396 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.452324    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:12:07.542933 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.542901    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-home\") pod \"5dede7bf-fe31-4550-8077-b1039eee0603\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") "
Aug 04 04:12:07.543097 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.542987    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-tmp-dir\") pod \"5dede7bf-fe31-4550-8077-b1039eee0603\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") "
Aug 04 04:12:07.543097 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.543022    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dede7bf-fe31-4550-8077-b1039eee0603-tls-certs\") pod \"5dede7bf-fe31-4550-8077-b1039eee0603\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") "
Aug 04 04:12:07.543097 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.543041    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nmst\" (UniqueName: \"kubernetes.io/projected/5dede7bf-fe31-4550-8077-b1039eee0603-kube-api-access-5nmst\") pod \"5dede7bf-fe31-4550-8077-b1039eee0603\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") "
Aug 04 04:12:07.543097 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.543058    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-dshm\") pod \"5dede7bf-fe31-4550-8077-b1039eee0603\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") "
Aug 04 04:12:07.543097 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.543096    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-model-cache\") pod \"5dede7bf-fe31-4550-8077-b1039eee0603\" (UID: \"5dede7bf-fe31-4550-8077-b1039eee0603\") "
Aug 04 04:12:07.543414 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.543242    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-home" (OuterVolumeSpecName: "home") pod "5dede7bf-fe31-4550-8077-b1039eee0603" (UID: "5dede7bf-fe31-4550-8077-b1039eee0603"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:12:07.543414 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.543258    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5dede7bf-fe31-4550-8077-b1039eee0603" (UID: "5dede7bf-fe31-4550-8077-b1039eee0603"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:12:07.543414 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.543388    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-home\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:07.543635 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.543430    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-tmp-dir\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:07.543635 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.543441    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-model-cache" (OuterVolumeSpecName: "model-cache") pod "5dede7bf-fe31-4550-8077-b1039eee0603" (UID: "5dede7bf-fe31-4550-8077-b1039eee0603"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:12:07.545497 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.545466    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-dshm" (OuterVolumeSpecName: "dshm") pod "5dede7bf-fe31-4550-8077-b1039eee0603" (UID: "5dede7bf-fe31-4550-8077-b1039eee0603"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:12:07.545623 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.545497    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dede7bf-fe31-4550-8077-b1039eee0603-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5dede7bf-fe31-4550-8077-b1039eee0603" (UID: "5dede7bf-fe31-4550-8077-b1039eee0603"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 04:12:07.545623 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.545515    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dede7bf-fe31-4550-8077-b1039eee0603-kube-api-access-5nmst" (OuterVolumeSpecName: "kube-api-access-5nmst") pod "5dede7bf-fe31-4550-8077-b1039eee0603" (UID: "5dede7bf-fe31-4550-8077-b1039eee0603"). InnerVolumeSpecName "kube-api-access-5nmst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:12:07.644678 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.644624    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dede7bf-fe31-4550-8077-b1039eee0603-tls-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:07.644678 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.644670    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nmst\" (UniqueName: \"kubernetes.io/projected/5dede7bf-fe31-4550-8077-b1039eee0603-kube-api-access-5nmst\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:07.644678 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.644681    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-dshm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:07.644678 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.644691    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dede7bf-fe31-4550-8077-b1039eee0603-model-cache\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:07.926384 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.926348    2559 generic.go:356] "Generic (PLEG): container finished" podID="5dede7bf-fe31-4550-8077-b1039eee0603" containerID="f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc" exitCode=0
Aug 04 04:12:07.926596 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.926396    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"
Aug 04 04:12:07.926596 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.926423    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt" event={"ID":"5dede7bf-fe31-4550-8077-b1039eee0603","Type":"ContainerDied","Data":"f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc"}
Aug 04 04:12:07.926596 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.926463    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt" event={"ID":"5dede7bf-fe31-4550-8077-b1039eee0603","Type":"ContainerDied","Data":"d3764998b2a36b5ad0842eea9a01a75f9bd46ed1c1661b39e43355cb88b32c9a"}
Aug 04 04:12:07.926596 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.926482    2559 scope.go:117] "RemoveContainer" containerID="f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc"
Aug 04 04:12:07.936509 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.936489    2559 scope.go:117] "RemoveContainer" containerID="f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc"
Aug 04 04:12:07.936852 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:12:07.936831    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc\": container with ID starting with f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc not found: ID does not exist" containerID="f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc"
Aug 04 04:12:07.936905 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.936861    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc"} err="failed to get container status \"f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc\": rpc error: code = NotFound desc = could not find container \"f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc\": container with ID starting with f44bc93b0e7cd497641763aa47530f0580e5aae5b08a15fc72f9db3aef7f0edc not found: ID does not exist"
Aug 04 04:12:07.949800 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.949766    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"]
Aug 04 04:12:07.951690 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:07.951659    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-881c6f/rate-test-model-881c6f-kserve-8bf6996dc-w54nt"]
Aug 04 04:12:08.260230 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.260193    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7436-140134711259184"]
Aug 04 04:12:08.260628 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.260600    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4cc7925-3779-4140-b76a-25d682b4c59f" containerName="main"
Aug 04 04:12:08.260628 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.260612    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4cc7925-3779-4140-b76a-25d682b4c59f" containerName="main"
Aug 04 04:12:08.260628 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.260625    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dede7bf-fe31-4550-8077-b1039eee0603" containerName="main"
Aug 04 04:12:08.260628 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.260631    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dede7bf-fe31-4550-8077-b1039eee0603" containerName="main"
Aug 04 04:12:08.260763 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.260687    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dede7bf-fe31-4550-8077-b1039eee0603" containerName="main"
Aug 04 04:12:08.260763 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.260695    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4cc7925-3779-4140-b76a-25d682b4c59f" containerName="main"
Aug 04 04:12:08.265103 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.265079    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7436-140134711259184"
Aug 04 04:12:08.267367 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.267316    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tdvwx\""
Aug 04 04:12:08.270210 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.270183    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7436-140134711259184"]
Aug 04 04:12:08.351440 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.351402    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjxt8\" (UniqueName: \"kubernetes.io/projected/f2611762-0e5f-4bc4-820a-1c904eb75f1d-kube-api-access-jjxt8\") pod \"test-curl-7436-140134711259184\" (UID: \"f2611762-0e5f-4bc4-820a-1c904eb75f1d\") " pod="opendatahub/test-curl-7436-140134711259184"
Aug 04 04:12:08.452617 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.452585    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjxt8\" (UniqueName: \"kubernetes.io/projected/f2611762-0e5f-4bc4-820a-1c904eb75f1d-kube-api-access-jjxt8\") pod \"test-curl-7436-140134711259184\" (UID: \"f2611762-0e5f-4bc4-820a-1c904eb75f1d\") " pod="opendatahub/test-curl-7436-140134711259184"
Aug 04 04:12:08.475611 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.475586    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjxt8\" (UniqueName: \"kubernetes.io/projected/f2611762-0e5f-4bc4-820a-1c904eb75f1d-kube-api-access-jjxt8\") pod \"test-curl-7436-140134711259184\" (UID: \"f2611762-0e5f-4bc4-820a-1c904eb75f1d\") " pod="opendatahub/test-curl-7436-140134711259184"
Aug 04 04:12:08.576101 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.575994    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7436-140134711259184"
Aug 04 04:12:08.731193 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.731156    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7436-140134711259184"]
Aug 04 04:12:08.733509 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:12:08.733482    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2611762_0e5f_4bc4_820a_1c904eb75f1d.slice/crio-4c39c26ee2f8f3b86e2b5ccdbcc5c9d8e936d7057b41063c31fa86765eb7c24f WatchSource:0}: Error finding container 4c39c26ee2f8f3b86e2b5ccdbcc5c9d8e936d7057b41063c31fa86765eb7c24f: Status 404 returned error can't find the container with id 4c39c26ee2f8f3b86e2b5ccdbcc5c9d8e936d7057b41063c31fa86765eb7c24f
Aug 04 04:12:08.734880 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.734860    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 04:12:08.933955 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:08.933857    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7436-140134711259184" event={"ID":"f2611762-0e5f-4bc4-820a-1c904eb75f1d","Type":"ContainerStarted","Data":"4c39c26ee2f8f3b86e2b5ccdbcc5c9d8e936d7057b41063c31fa86765eb7c24f"}
Aug 04 04:12:09.239742 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:09.239709    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dede7bf-fe31-4550-8077-b1039eee0603" path="/var/lib/kubelet/pods/5dede7bf-fe31-4550-8077-b1039eee0603/volumes"
Aug 04 04:12:09.941861 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:09.941760    2559 generic.go:356] "Generic (PLEG): container finished" podID="f2611762-0e5f-4bc4-820a-1c904eb75f1d" containerID="86867b7a3319ee1cfdc87ccd8bf38367230aed91b6a3719305128108ad255f18" exitCode=0
Aug 04 04:12:09.941861 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:09.941803    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7436-140134711259184" event={"ID":"f2611762-0e5f-4bc4-820a-1c904eb75f1d","Type":"ContainerDied","Data":"86867b7a3319ee1cfdc87ccd8bf38367230aed91b6a3719305128108ad255f18"}
Aug 04 04:12:11.081739 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.081715    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7436-140134711259184"
Aug 04 04:12:11.102603 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.102574    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7436-140134711259184_f2611762-0e5f-4bc4-820a-1c904eb75f1d/test-curl-7436-140134711259184/0.log"
Aug 04 04:12:11.126240 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.126192    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7436-140134711259184"]
Aug 04 04:12:11.128523 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.128494    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7436-140134711259184"]
Aug 04 04:12:11.179256 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.179216    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjxt8\" (UniqueName: \"kubernetes.io/projected/f2611762-0e5f-4bc4-820a-1c904eb75f1d-kube-api-access-jjxt8\") pod \"f2611762-0e5f-4bc4-820a-1c904eb75f1d\" (UID: \"f2611762-0e5f-4bc4-820a-1c904eb75f1d\") "
Aug 04 04:12:11.181515 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.181484    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2611762-0e5f-4bc4-820a-1c904eb75f1d-kube-api-access-jjxt8" (OuterVolumeSpecName: "kube-api-access-jjxt8") pod "f2611762-0e5f-4bc4-820a-1c904eb75f1d" (UID: "f2611762-0e5f-4bc4-820a-1c904eb75f1d"). InnerVolumeSpecName "kube-api-access-jjxt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:12:11.238144 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.238111    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2611762-0e5f-4bc4-820a-1c904eb75f1d" path="/var/lib/kubelet/pods/f2611762-0e5f-4bc4-820a-1c904eb75f1d/volumes"
Aug 04 04:12:11.280559 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.280513    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjxt8\" (UniqueName: \"kubernetes.io/projected/f2611762-0e5f-4bc4-820a-1c904eb75f1d-kube-api-access-jjxt8\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:12:11.660822 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.660719    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:11.958481 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.958387    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"]
Aug 04 04:12:11.958884 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.958866    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2611762-0e5f-4bc4-820a-1c904eb75f1d" containerName="test-curl-7436-140134711259184"
Aug 04 04:12:11.958950 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.958888    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2611762-0e5f-4bc4-820a-1c904eb75f1d" containerName="test-curl-7436-140134711259184"
Aug 04 04:12:11.958996 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.958989    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2611762-0e5f-4bc4-820a-1c904eb75f1d" containerName="test-curl-7436-140134711259184"
Aug 04 04:12:11.959867 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.959844    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7436-140134711259184"
Aug 04 04:12:11.962380 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.962358    2559 scope.go:117] "RemoveContainer" containerID="86867b7a3319ee1cfdc87ccd8bf38367230aed91b6a3719305128108ad255f18"
Aug 04 04:12:11.962726 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.962682    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:11.965067 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.965041    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"]
Aug 04 04:12:11.965601 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.965527    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-2d8d0c-openshift-default-dockercfg-zlddq\""
Aug 04 04:12:11.974151 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:11.974125    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:12.074827 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.074795    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:12.087221 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.087185    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-podinfo\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.087624 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.087230    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-socket\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.087624 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.087265    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-credential-socket\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.087624 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.087317    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istiod-ca-cert\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.087624 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.087413    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-envoy\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.087624 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.087443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-token\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.087624 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.087469    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-data\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.087624 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.087511    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-certs\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.087624 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.087529    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvhq5\" (UniqueName: \"kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-kube-api-access-kvhq5\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.188985 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.188938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-envoy\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.188985 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.188988    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-token\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.189326 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.189006    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-data\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.189326 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.189037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-certs\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.189326 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.189063    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvhq5\" (UniqueName: \"kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-kube-api-access-kvhq5\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.189326 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.189108    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-podinfo\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.189326 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.189134    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-socket\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.189326 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.189303    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-credential-socket\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.189696 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.189359    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istiod-ca-cert\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.189696 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.189479    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-data\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.189696 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.189504    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-certs\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.189863 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.189710    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-socket\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.189863 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.189726    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-credential-socket\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.190068 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.190045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istiod-ca-cert\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.191517 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.191494    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-envoy\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.191678 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.191658    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-podinfo\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.197261 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.197233    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-token\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.197383 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.197363    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvhq5\" (UniqueName: \"kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-kube-api-access-kvhq5\") pod \"e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") " pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.277497 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.277454    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:12.427986 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.427952    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"]
Aug 04 04:12:12.431192 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:12:12.431151    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc4f2a46_6e83_4f51_8d03_bbc3b90b6c3a.slice/crio-4c5207410cf5aa9a10b6c3f7d53c06901e0113e3f8fc81fb745757a7283a330f WatchSource:0}: Error finding container 4c5207410cf5aa9a10b6c3f7d53c06901e0113e3f8fc81fb745757a7283a330f: Status 404 returned error can't find the container with id 4c5207410cf5aa9a10b6c3f7d53c06901e0113e3f8fc81fb745757a7283a330f
Aug 04 04:12:12.434112 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.434056    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:12:12.434245 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.434166    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:12:12.434245 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.434213    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:12:12.966640 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.966597    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" event={"ID":"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a","Type":"ContainerStarted","Data":"7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7"}
Aug 04 04:12:12.966859 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.966665    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" event={"ID":"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a","Type":"ContainerStarted","Data":"4c5207410cf5aa9a10b6c3f7d53c06901e0113e3f8fc81fb745757a7283a330f"}
Aug 04 04:12:12.992449 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:12.992395    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" podStartSLOduration=1.992378044 podStartE2EDuration="1.992378044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:12:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:12:12.990236406 +0000 UTC m=+3750.288376894" watchObservedRunningTime="2026-08-04 04:12:12.992378044 +0000 UTC m=+3750.290518531"
Aug 04 04:12:13.278161 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:13.278123    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:13.283325 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:13.283295    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:13.973322 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:13.973293    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:13.974320 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:13.974294    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:15.030129 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:15.030095    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:15.155394 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:15.155342    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:16.087516 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:16.087464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:16.373968 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:16.373872    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:16.699515 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:16.699416    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:16.798217 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:16.798170    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:33.084248 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:33.084215    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerID="7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7" exitCode=0
Aug 04 04:12:33.084713 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:33.084285    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" event={"ID":"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a","Type":"ContainerDied","Data":"7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7"}
Aug 04 04:12:33.084713 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:33.084707    2559 scope.go:117] "RemoveContainer" containerID="7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7"
Aug 04 04:12:33.085789 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:33.085755    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:12:33.085900 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:33.085845    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:12:33.085965 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:33.085918    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:12:34.091478 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:34.091441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" event={"ID":"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a","Type":"ContainerStarted","Data":"f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8"}
Aug 04 04:12:34.278249 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:34.278204    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:34.278452 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:34.278427    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Aug 04 04:12:34.278514 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:34.278477    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Aug 04 04:12:35.278613 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:35.278492    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Aug 04 04:12:35.278613 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:35.278587    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Aug 04 04:12:36.278700 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:36.278655    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Aug 04 04:12:36.279081 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:36.278723    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Aug 04 04:12:37.278758 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:37.278723    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Aug 04 04:12:37.279170 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:37.278784    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Aug 04 04:12:38.278853 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:38.278817    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Aug 04 04:12:38.279256 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:38.278874    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Aug 04 04:12:39.282872 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:39.282836    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:39.283362 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:39.283213    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:39.283972 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:39.283948    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:12:39.974966 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:39.974931    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:40.730554 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:40.730512    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:41.001389 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.001353    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"]
Aug 04 04:12:41.003991 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.003966    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.006666 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.006641    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-b0089f-openshift-default-dockercfg-n6s9w\""
Aug 04 04:12:41.019012 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.018979    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"]
Aug 04 04:12:41.047393 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.047360    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:41.077679 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.077636    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-token\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.077679 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.077680    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-envoy\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.077888 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.077760    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-certs\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.077888 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.077796    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-data\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.077888 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.077878    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-socket\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.077996 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.077928    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-podinfo\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.078032 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.078003    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-credential-socket\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.078067 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.078051    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/583aca4d-1892-47a6-8f18-d4f1892ad8da-istiod-ca-cert\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.078101 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.078075    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkxj2\" (UniqueName: \"kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-kube-api-access-nkxj2\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.179168 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179122    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-socket\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.179386 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179187    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-podinfo\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.179386 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179218    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-credential-socket\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.179386 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179258    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/583aca4d-1892-47a6-8f18-d4f1892ad8da-istiod-ca-cert\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.179386 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179280    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkxj2\" (UniqueName: \"kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-kube-api-access-nkxj2\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.179386 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-token\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.179386 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179350    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-envoy\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.179790 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-certs\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.179790 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179431    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-data\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.179790 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179650    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-socket\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.179790 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179693    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-credential-socket\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.180022 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179869    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-data\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.180022 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.179956    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-certs\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.180152 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.180125    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/583aca4d-1892-47a6-8f18-d4f1892ad8da-istiod-ca-cert\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.181712 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.181690    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-envoy\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.181941 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.181924    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-podinfo\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.188665 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.188630    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-token\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.188835 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.188817    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkxj2\" (UniqueName: \"kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-kube-api-access-nkxj2\") pod \"e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") " pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.315507 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.315411    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:41.459628 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.459595    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"]
Aug 04 04:12:41.461180 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:12:41.461151    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod583aca4d_1892_47a6_8f18_d4f1892ad8da.slice/crio-65aee3c3795100b84ddac4c5a091a20ffccff74f073457620f0ff2da7a5d5721 WatchSource:0}: Error finding container 65aee3c3795100b84ddac4c5a091a20ffccff74f073457620f0ff2da7a5d5721: Status 404 returned error can't find the container with id 65aee3c3795100b84ddac4c5a091a20ffccff74f073457620f0ff2da7a5d5721
Aug 04 04:12:41.463617 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.463534    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:12:41.463706 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.463669    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:12:41.463750 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.463707    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:12:41.626672 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:41.626534    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:42.135793 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:42.135756    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" event={"ID":"583aca4d-1892-47a6-8f18-d4f1892ad8da","Type":"ContainerStarted","Data":"fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3"}
Aug 04 04:12:42.136206 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:42.135809    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" event={"ID":"583aca4d-1892-47a6-8f18-d4f1892ad8da","Type":"ContainerStarted","Data":"65aee3c3795100b84ddac4c5a091a20ffccff74f073457620f0ff2da7a5d5721"}
Aug 04 04:12:42.156903 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:42.156851    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" podStartSLOduration=2.156829786 podStartE2EDuration="2.156829786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:12:42.155890619 +0000 UTC m=+3779.454031128" watchObservedRunningTime="2026-08-04 04:12:42.156829786 +0000 UTC m=+3779.454970278"
Aug 04 04:12:42.316643 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:42.316604    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:42.322276 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:42.322247    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:43.140622 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:43.140584    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:43.141649 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:43.141630    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:12:44.121416 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:44.121318    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:44.222759 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:44.222727    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:45.377195 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:45.377158    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:45.691568 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:45.691458    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:45.870980 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:45.870948    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:12:45.962644 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:12:45.962522    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:13:03.243747 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:03.243711    2559 generic.go:356] "Generic (PLEG): container finished" podID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerID="fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3" exitCode=0
Aug 04 04:13:03.244184 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:03.243781    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" event={"ID":"583aca4d-1892-47a6-8f18-d4f1892ad8da","Type":"ContainerDied","Data":"fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3"}
Aug 04 04:13:03.244258 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:03.244240    2559 scope.go:117] "RemoveContainer" containerID="fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3"
Aug 04 04:13:03.245356 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:03.245328    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:13:03.245426 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:03.245400    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:13:03.245468 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:03.245433    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Aug 04 04:13:04.250856 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:04.250824    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" event={"ID":"583aca4d-1892-47a6-8f18-d4f1892ad8da","Type":"ContainerStarted","Data":"deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b"}
Aug 04 04:13:04.315823 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:04.315779    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:13:04.316049 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:04.316028    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Aug 04 04:13:04.316130 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:04.316083    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Aug 04 04:13:05.316152 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:05.316109    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Aug 04 04:13:05.316587 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:05.316173    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Aug 04 04:13:06.315942 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:06.315902    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Aug 04 04:13:06.316138 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:06.315975    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Aug 04 04:13:07.316131 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:07.316091    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Aug 04 04:13:07.316521 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:07.316155    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Aug 04 04:13:08.316411 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:08.316369    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Aug 04 04:13:08.316821 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:08.316432    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Aug 04 04:13:09.168853 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:09.168819    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:13:09.320157 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:09.320124    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:13:09.320664 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:09.320526    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:13:09.321158 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:09.321137    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:13:19.785656 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.785620    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"]
Aug 04 04:13:19.788135 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.788119    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.791633 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.791604    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-2d8d0c\"/\"kube-root-ca.crt\""
Aug 04 04:13:19.791796 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.791642    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-2d8d0c\"/\"openshift-service-ca.crt\""
Aug 04 04:13:19.791796 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.791721    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-2d8d0c\"/\"ipp-route-2d8d0c-kserve-self-signed-certs\""
Aug 04 04:13:19.791796 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.791730    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-2d8d0c\"/\"default-dockercfg-7g4z4\""
Aug 04 04:13:19.803333 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.803303    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"]
Aug 04 04:13:19.831228 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.831195    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:13:19.886849 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.886809    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-dshm\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.887028 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.886932    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-tmp-dir\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.887028 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.886962    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b5a616e-9a87-4b93-b13c-755df2463220-tls-certs\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.887107 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.887039    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxtdg\" (UniqueName: \"kubernetes.io/projected/4b5a616e-9a87-4b93-b13c-755df2463220-kube-api-access-xxtdg\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.887107 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.887067    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-home\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.887107 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.887088    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-model-cache\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.987961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.987926    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-tmp-dir\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.988151 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.987970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b5a616e-9a87-4b93-b13c-755df2463220-tls-certs\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.988151 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.988007    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxtdg\" (UniqueName: \"kubernetes.io/projected/4b5a616e-9a87-4b93-b13c-755df2463220-kube-api-access-xxtdg\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.988151 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.988026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-home\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.988151 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.988042    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-model-cache\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.988151 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.988081    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-dshm\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.988441 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.988388    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-tmp-dir\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.988441 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.988412    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-home\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.988580 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.988440    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-model-cache\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.990433 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.990404    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-dshm\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.990716 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.990693    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b5a616e-9a87-4b93-b13c-755df2463220-tls-certs\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:19.996227 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:19.996205    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxtdg\" (UniqueName: \"kubernetes.io/projected/4b5a616e-9a87-4b93-b13c-755df2463220-kube-api-access-xxtdg\") pod \"ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") " pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:20.099020 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:20.098921    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:20.250844 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:20.250815    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"]
Aug 04 04:13:20.252743 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:13:20.252716    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b5a616e_9a87_4b93_b13c_755df2463220.slice/crio-49c5a3b693cba4cf589751690807383078139319fdd5c2a570eb07b1ab67108a WatchSource:0}: Error finding container 49c5a3b693cba4cf589751690807383078139319fdd5c2a570eb07b1ab67108a: Status 404 returned error can't find the container with id 49c5a3b693cba4cf589751690807383078139319fdd5c2a570eb07b1ab67108a
Aug 04 04:13:20.349098 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:20.349003    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct" event={"ID":"4b5a616e-9a87-4b93-b13c-755df2463220","Type":"ContainerStarted","Data":"2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c"}
Aug 04 04:13:20.349098 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:20.349051    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct" event={"ID":"4b5a616e-9a87-4b93-b13c-755df2463220","Type":"ContainerStarted","Data":"49c5a3b693cba4cf589751690807383078139319fdd5c2a570eb07b1ab67108a"}
Aug 04 04:13:20.366984 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:20.366923    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct" podStartSLOduration=1.366907732 podStartE2EDuration="1.366907732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:13:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:13:20.365194664 +0000 UTC m=+3817.663335152" watchObservedRunningTime="2026-08-04 04:13:20.366907732 +0000 UTC m=+3817.665048220"
Aug 04 04:13:30.099826 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:30.099780    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:30.100260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:30.099853    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:30.112721 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:30.112695    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:30.414443 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:30.414350    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:13:40.998997 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:40.998958    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:13:41.100046 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:13:41.100013    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:07.235317 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:07.235214    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:07.586517 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:07.586432    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:07.840484 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:07.840370    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:13.766939 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:13.766898    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:13.862558 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:13.862520    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:14.924884 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:14.924849    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:15.412524 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.412489    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:15.512139 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.512107    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:15.517442 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.517413    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"]
Aug 04 04:14:15.519851 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.519804    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:15.524065 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.524043    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-feaa4d\""
Aug 04 04:14:15.527029 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.527007    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-feaa4d-dockercfg-vm9rj\""
Aug 04 04:14:15.534898 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.534865    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"]
Aug 04 04:14:15.640164 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.640124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7774e93f-75fa-4fc7-9dee-0a61a5945edd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk\" (UID: \"7774e93f-75fa-4fc7-9dee-0a61a5945edd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:15.640460 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.640437    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wksq\" (UniqueName: \"kubernetes.io/projected/7774e93f-75fa-4fc7-9dee-0a61a5945edd-kube-api-access-4wksq\") pod \"payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk\" (UID: \"7774e93f-75fa-4fc7-9dee-0a61a5945edd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:15.741411 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.741371    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7774e93f-75fa-4fc7-9dee-0a61a5945edd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk\" (UID: \"7774e93f-75fa-4fc7-9dee-0a61a5945edd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:15.741607 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.741437    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wksq\" (UniqueName: \"kubernetes.io/projected/7774e93f-75fa-4fc7-9dee-0a61a5945edd-kube-api-access-4wksq\") pod \"payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk\" (UID: \"7774e93f-75fa-4fc7-9dee-0a61a5945edd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:15.742181 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.742159    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7774e93f-75fa-4fc7-9dee-0a61a5945edd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk\" (UID: \"7774e93f-75fa-4fc7-9dee-0a61a5945edd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:15.749885 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.749858    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wksq\" (UniqueName: \"kubernetes.io/projected/7774e93f-75fa-4fc7-9dee-0a61a5945edd-kube-api-access-4wksq\") pod \"payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk\" (UID: \"7774e93f-75fa-4fc7-9dee-0a61a5945edd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:15.832378 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.832336    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:15.974750 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:15.974716    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"]
Aug 04 04:14:15.975090 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:14:15.974866    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7774e93f_75fa_4fc7_9dee_0a61a5945edd.slice/crio-5103b2e2802b75c093360ec9830223d2b2d4af42aaec65adcddbee4eee23c045 WatchSource:0}: Error finding container 5103b2e2802b75c093360ec9830223d2b2d4af42aaec65adcddbee4eee23c045: Status 404 returned error can't find the container with id 5103b2e2802b75c093360ec9830223d2b2d4af42aaec65adcddbee4eee23c045
Aug 04 04:14:16.654211 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:16.654176    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk" event={"ID":"7774e93f-75fa-4fc7-9dee-0a61a5945edd","Type":"ContainerStarted","Data":"6009ac1d1e82388b96bc0ccf060e60467c4747000dc602dd450415fccf2b6f2a"}
Aug 04 04:14:16.654439 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:16.654231    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk" event={"ID":"7774e93f-75fa-4fc7-9dee-0a61a5945edd","Type":"ContainerStarted","Data":"5103b2e2802b75c093360ec9830223d2b2d4af42aaec65adcddbee4eee23c045"}
Aug 04 04:14:16.654439 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:16.654284    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:16.672445 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:16.672395    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk" podStartSLOduration=1.672376138 podStartE2EDuration="1.672376138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:14:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:14:16.670351078 +0000 UTC m=+3873.968491627" watchObservedRunningTime="2026-08-04 04:14:16.672376138 +0000 UTC m=+3873.970516625"
Aug 04 04:14:27.662415 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:27.662382    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:38.814375 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:38.814334    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:45.564315 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:45.564261    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:45.600805 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:45.600532    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"]
Aug 04 04:14:45.601000 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:45.600908    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk" podUID="7774e93f-75fa-4fc7-9dee-0a61a5945edd" containerName="payload-pre-processing" containerID="cri-o://6009ac1d1e82388b96bc0ccf060e60467c4747000dc602dd450415fccf2b6f2a" gracePeriod=30
Aug 04 04:14:45.660818 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:45.660768    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:45.751453 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:45.751415    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:45.805115 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:45.805083    2559 generic.go:356] "Generic (PLEG): container finished" podID="7774e93f-75fa-4fc7-9dee-0a61a5945edd" containerID="6009ac1d1e82388b96bc0ccf060e60467c4747000dc602dd450415fccf2b6f2a" exitCode=0
Aug 04 04:14:45.805266 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:45.805137    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk" event={"ID":"7774e93f-75fa-4fc7-9dee-0a61a5945edd","Type":"ContainerDied","Data":"6009ac1d1e82388b96bc0ccf060e60467c4747000dc602dd450415fccf2b6f2a"}
Aug 04 04:14:45.834114 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:45.834014    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:45.876947 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:45.876917    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:46.041692 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.041663    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7774e93f-75fa-4fc7-9dee-0a61a5945edd-plugins-config-volume\") pod \"7774e93f-75fa-4fc7-9dee-0a61a5945edd\" (UID: \"7774e93f-75fa-4fc7-9dee-0a61a5945edd\") "
Aug 04 04:14:46.041855 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.041704    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wksq\" (UniqueName: \"kubernetes.io/projected/7774e93f-75fa-4fc7-9dee-0a61a5945edd-kube-api-access-4wksq\") pod \"7774e93f-75fa-4fc7-9dee-0a61a5945edd\" (UID: \"7774e93f-75fa-4fc7-9dee-0a61a5945edd\") "
Aug 04 04:14:46.042112 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.042089    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7774e93f-75fa-4fc7-9dee-0a61a5945edd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7774e93f-75fa-4fc7-9dee-0a61a5945edd" (UID: "7774e93f-75fa-4fc7-9dee-0a61a5945edd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 04:14:46.043881 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.043850    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7774e93f-75fa-4fc7-9dee-0a61a5945edd-kube-api-access-4wksq" (OuterVolumeSpecName: "kube-api-access-4wksq") pod "7774e93f-75fa-4fc7-9dee-0a61a5945edd" (UID: "7774e93f-75fa-4fc7-9dee-0a61a5945edd"). InnerVolumeSpecName "kube-api-access-4wksq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:14:46.127520 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.127432    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:46.143106 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.143073    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7774e93f-75fa-4fc7-9dee-0a61a5945edd-plugins-config-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:14:46.143106 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.143102    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wksq\" (UniqueName: \"kubernetes.io/projected/7774e93f-75fa-4fc7-9dee-0a61a5945edd-kube-api-access-4wksq\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:14:46.811146 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.811118    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"
Aug 04 04:14:46.811590 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.811144    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk" event={"ID":"7774e93f-75fa-4fc7-9dee-0a61a5945edd","Type":"ContainerDied","Data":"5103b2e2802b75c093360ec9830223d2b2d4af42aaec65adcddbee4eee23c045"}
Aug 04 04:14:46.811590 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.811200    2559 scope.go:117] "RemoveContainer" containerID="6009ac1d1e82388b96bc0ccf060e60467c4747000dc602dd450415fccf2b6f2a"
Aug 04 04:14:46.837058 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.837021    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"]
Aug 04 04:14:46.841351 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:46.841325    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-feaa4d-68b4d8f6d4-rl7gk"]
Aug 04 04:14:47.237895 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:47.237853    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7774e93f-75fa-4fc7-9dee-0a61a5945edd" path="/var/lib/kubelet/pods/7774e93f-75fa-4fc7-9dee-0a61a5945edd/volumes"
Aug 04 04:14:48.873594 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:48.873558    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:52.260744 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:52.260707    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:52.326704 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:52.326588    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:52.533213 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:52.533084    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:14:52.786716 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:14:52.786627    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:15:04.398459 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:04.398420    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs"]
Aug 04 04:15:04.402935 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:04.402908    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29763570-wfncs"]
Aug 04 04:15:05.239877 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:05.239839    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69933532-cdc4-43a2-a946-17121b7c7053" path="/var/lib/kubelet/pods/69933532-cdc4-43a2-a946-17121b7c7053/volumes"
Aug 04 04:15:11.695489 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:11.695454    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:15:11.821867 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:11.821821    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:15:11.926778 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:11.926736    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:15:11.994895 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:11.994860    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:15:12.433056 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:12.432974    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:15:17.401451 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:17.401409    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"]
Aug 04 04:15:17.497318 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:17.497276    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:15:19.780854 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:19.780803    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:15:32.421077 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.421023    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct" podUID="4b5a616e-9a87-4b93-b13c-755df2463220" containerName="main" containerID="cri-o://2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c" gracePeriod=45
Aug 04 04:15:32.668211 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.668183    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:15:32.694461 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.694368    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-tmp-dir\") pod \"4b5a616e-9a87-4b93-b13c-755df2463220\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") "
Aug 04 04:15:32.694461 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.694418    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-dshm\") pod \"4b5a616e-9a87-4b93-b13c-755df2463220\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") "
Aug 04 04:15:32.694707 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.694520    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-home\") pod \"4b5a616e-9a87-4b93-b13c-755df2463220\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") "
Aug 04 04:15:32.694707 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.694581    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-model-cache\") pod \"4b5a616e-9a87-4b93-b13c-755df2463220\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") "
Aug 04 04:15:32.694707 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.694643    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxtdg\" (UniqueName: \"kubernetes.io/projected/4b5a616e-9a87-4b93-b13c-755df2463220-kube-api-access-xxtdg\") pod \"4b5a616e-9a87-4b93-b13c-755df2463220\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") "
Aug 04 04:15:32.694707 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.694670    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b5a616e-9a87-4b93-b13c-755df2463220-tls-certs\") pod \"4b5a616e-9a87-4b93-b13c-755df2463220\" (UID: \"4b5a616e-9a87-4b93-b13c-755df2463220\") "
Aug 04 04:15:32.694916 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.694689    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b5a616e-9a87-4b93-b13c-755df2463220" (UID: "4b5a616e-9a87-4b93-b13c-755df2463220"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:15:32.694916 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.694778    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-home" (OuterVolumeSpecName: "home") pod "4b5a616e-9a87-4b93-b13c-755df2463220" (UID: "4b5a616e-9a87-4b93-b13c-755df2463220"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:15:32.695018 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.694907    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b5a616e-9a87-4b93-b13c-755df2463220" (UID: "4b5a616e-9a87-4b93-b13c-755df2463220"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:15:32.695071 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.695062    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-home\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:15:32.695124 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.695082    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-model-cache\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:15:32.695124 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.695099    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-tmp-dir\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:15:32.697393 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.697360    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b5a616e-9a87-4b93-b13c-755df2463220-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b5a616e-9a87-4b93-b13c-755df2463220" (UID: "4b5a616e-9a87-4b93-b13c-755df2463220"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 04:15:32.697533 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.697434    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b5a616e-9a87-4b93-b13c-755df2463220-kube-api-access-xxtdg" (OuterVolumeSpecName: "kube-api-access-xxtdg") pod "4b5a616e-9a87-4b93-b13c-755df2463220" (UID: "4b5a616e-9a87-4b93-b13c-755df2463220"). InnerVolumeSpecName "kube-api-access-xxtdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:15:32.697533 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.697457    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-dshm" (OuterVolumeSpecName: "dshm") pod "4b5a616e-9a87-4b93-b13c-755df2463220" (UID: "4b5a616e-9a87-4b93-b13c-755df2463220"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:15:32.795965 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.795930    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxtdg\" (UniqueName: \"kubernetes.io/projected/4b5a616e-9a87-4b93-b13c-755df2463220-kube-api-access-xxtdg\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:15:32.795965 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.795962    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b5a616e-9a87-4b93-b13c-755df2463220-tls-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:15:32.795965 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:32.795971    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b5a616e-9a87-4b93-b13c-755df2463220-dshm\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:15:33.057337 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:33.057300    2559 generic.go:356] "Generic (PLEG): container finished" podID="4b5a616e-9a87-4b93-b13c-755df2463220" containerID="2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c" exitCode=0
Aug 04 04:15:33.057573 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:33.057343    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"
Aug 04 04:15:33.057573 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:33.057364    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct" event={"ID":"4b5a616e-9a87-4b93-b13c-755df2463220","Type":"ContainerDied","Data":"2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c"}
Aug 04 04:15:33.057573 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:33.057404    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct" event={"ID":"4b5a616e-9a87-4b93-b13c-755df2463220","Type":"ContainerDied","Data":"49c5a3b693cba4cf589751690807383078139319fdd5c2a570eb07b1ab67108a"}
Aug 04 04:15:33.057573 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:33.057426    2559 scope.go:117] "RemoveContainer" containerID="2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c"
Aug 04 04:15:33.066705 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:33.066685    2559 scope.go:117] "RemoveContainer" containerID="2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c"
Aug 04 04:15:33.066985 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:15:33.066964    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c\": container with ID starting with 2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c not found: ID does not exist" containerID="2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c"
Aug 04 04:15:33.067056 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:33.066993    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c"} err="failed to get container status \"2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c\": rpc error: code = NotFound desc = could not find container \"2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c\": container with ID starting with 2da77ef142266e9dc11f5437e4633646d2422aab6dd89343eb976f5dfdac113c not found: ID does not exist"
Aug 04 04:15:33.078925 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:33.078893    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"]
Aug 04 04:15:33.082807 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:33.082776    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-2d8d0c/ipp-route-2d8d0c-kserve-6ffb85b4f5-78gct"]
Aug 04 04:15:33.239308 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:33.239265    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b5a616e-9a87-4b93-b13c-755df2463220" path="/var/lib/kubelet/pods/4b5a616e-9a87-4b93-b13c-755df2463220/volumes"
Aug 04 04:15:43.792519 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:43.792437    2559 scope.go:117] "RemoveContainer" containerID="c4f9d1f0c56f9194a44f83c2933e07ddcff08e7f9921e89ca0a6f55074a6c9ac"
Aug 04 04:15:54.819415 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:54.819358    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:15:55.094109 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:15:55.094017    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:00.608066 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:00.608025    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:00.710153 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:00.710117    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:00.882862 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:00.882774    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"]
Aug 04 04:16:00.883067 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:00.883039    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy" containerID="cri-o://f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8" gracePeriod=30
Aug 04 04:16:00.902280 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:00.902247    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:01.116261 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:01.116229    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:06.144003 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.143927    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:16:06.235179 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.235144    2559 generic.go:356] "Generic (PLEG): container finished" podID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerID="f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8" exitCode=0
Aug 04 04:16:06.235367 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.235191    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"
Aug 04 04:16:06.235367 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.235223    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" event={"ID":"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a","Type":"ContainerDied","Data":"f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8"}
Aug 04 04:16:06.235367 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.235275    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv" event={"ID":"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a","Type":"ContainerDied","Data":"4c5207410cf5aa9a10b6c3f7d53c06901e0113e3f8fc81fb745757a7283a330f"}
Aug 04 04:16:06.235367 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.235297    2559 scope.go:117] "RemoveContainer" containerID="f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8"
Aug 04 04:16:06.244102 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.244081    2559 scope.go:117] "RemoveContainer" containerID="7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7"
Aug 04 04:16:06.247421 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.247398    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-podinfo\") pod \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") "
Aug 04 04:16:06.247499 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.247436    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-certs\") pod \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") "
Aug 04 04:16:06.247499 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.247485    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-envoy\") pod \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") "
Aug 04 04:16:06.247590 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.247514    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istiod-ca-cert\") pod \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") "
Aug 04 04:16:06.247590 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.247560    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-data\") pod \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") "
Aug 04 04:16:06.247688 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.247598    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-socket\") pod \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") "
Aug 04 04:16:06.247688 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.247653    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvhq5\" (UniqueName: \"kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-kube-api-access-kvhq5\") pod \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") "
Aug 04 04:16:06.247795 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.247711    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-token\") pod \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") "
Aug 04 04:16:06.247795 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.247737    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-credential-socket\") pod \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\" (UID: \"bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a\") "
Aug 04 04:16:06.247904 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.247776    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" (UID: "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:06.248130 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.248002    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" (UID: "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:06.248130 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.248071    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:06.248274 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.248236    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" (UID: "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 04:16:06.248362 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.248338    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" (UID: "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:06.248522 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.248492    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-data" (OuterVolumeSpecName: "istio-data") pod "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" (UID: "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:06.249982 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.249960    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" (UID: "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:06.250082 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.249983    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" (UID: "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 04:16:06.250239 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.250220    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-kube-api-access-kvhq5" (OuterVolumeSpecName: "kube-api-access-kvhq5") pod "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" (UID: "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a"). InnerVolumeSpecName "kube-api-access-kvhq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:16:06.250304 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.250247    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-token" (OuterVolumeSpecName: "istio-token") pod "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" (UID: "bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:16:06.253427 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.253407    2559 scope.go:117] "RemoveContainer" containerID="f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8"
Aug 04 04:16:06.253783 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:16:06.253759    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8\": container with ID starting with f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8 not found: ID does not exist" containerID="f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8"
Aug 04 04:16:06.253852 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.253793    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8"} err="failed to get container status \"f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8\": rpc error: code = NotFound desc = could not find container \"f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8\": container with ID starting with f054c4ced14c47cab0f42070c89297afacb415efa1f1b4367b68e955379550b8 not found: ID does not exist"
Aug 04 04:16:06.253852 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.253811    2559 scope.go:117] "RemoveContainer" containerID="7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7"
Aug 04 04:16:06.254078 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:16:06.254058    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7\": container with ID starting with 7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7 not found: ID does not exist" containerID="7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7"
Aug 04 04:16:06.254122 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.254084    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7"} err="failed to get container status \"7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7\": rpc error: code = NotFound desc = could not find container \"7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7\": container with ID starting with 7430d0f187b2a8416bdea22522b5c25b704fcbbef048b69d83279f4c065bfdd7 not found: ID does not exist"
Aug 04 04:16:06.349667 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.349628    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:06.349667 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.349661    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:06.349667 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.349670    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:06.349667 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.349678    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:06.349961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.349685    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:06.349961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.349694    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvhq5\" (UniqueName: \"kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-kube-api-access-kvhq5\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:06.349961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.349704    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:06.349961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.349712    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:06.558314 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.558279    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"]
Aug 04 04:16:06.562445 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:06.562415    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-2d8d0c-openshift-default-7c684cdc4d-57hcv"]
Aug 04 04:16:07.238866 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:07.238830    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" path="/var/lib/kubelet/pods/bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a/volumes"
Aug 04 04:16:11.729343 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:11.729289    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:11.826244 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:11.826209    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:11.898653 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:11.898603    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:11.953632 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:11.953593    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:12.266174 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:12.266136    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:18.276779 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:18.276733    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:18.345452 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:18.345408    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:18.467877 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:18.467842    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"]
Aug 04 04:16:18.468072 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:18.468051    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy" containerID="cri-o://deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b" gracePeriod=30
Aug 04 04:16:18.485016 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:18.484980    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:18.671506 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:18.671419    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:23.733529 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.733494    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:16:23.816644 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.816607    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-envoy\") pod \"583aca4d-1892-47a6-8f18-d4f1892ad8da\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") "
Aug 04 04:16:23.816841 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.816662    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/583aca4d-1892-47a6-8f18-d4f1892ad8da-istiod-ca-cert\") pod \"583aca4d-1892-47a6-8f18-d4f1892ad8da\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") "
Aug 04 04:16:23.816841 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.816696    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-certs\") pod \"583aca4d-1892-47a6-8f18-d4f1892ad8da\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") "
Aug 04 04:16:23.816841 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.816760    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-data\") pod \"583aca4d-1892-47a6-8f18-d4f1892ad8da\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") "
Aug 04 04:16:23.816841 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.816779    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-socket\") pod \"583aca4d-1892-47a6-8f18-d4f1892ad8da\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") "
Aug 04 04:16:23.816841 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.816813    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-credential-socket\") pod \"583aca4d-1892-47a6-8f18-d4f1892ad8da\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") "
Aug 04 04:16:23.816841 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.816831    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkxj2\" (UniqueName: \"kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-kube-api-access-nkxj2\") pod \"583aca4d-1892-47a6-8f18-d4f1892ad8da\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") "
Aug 04 04:16:23.817164 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.816860    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-token\") pod \"583aca4d-1892-47a6-8f18-d4f1892ad8da\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") "
Aug 04 04:16:23.817164 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.816894    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-podinfo\") pod \"583aca4d-1892-47a6-8f18-d4f1892ad8da\" (UID: \"583aca4d-1892-47a6-8f18-d4f1892ad8da\") "
Aug 04 04:16:23.817164 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.817061    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "583aca4d-1892-47a6-8f18-d4f1892ad8da" (UID: "583aca4d-1892-47a6-8f18-d4f1892ad8da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:23.817164 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.817099    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/583aca4d-1892-47a6-8f18-d4f1892ad8da-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "583aca4d-1892-47a6-8f18-d4f1892ad8da" (UID: "583aca4d-1892-47a6-8f18-d4f1892ad8da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 04:16:23.817367 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.817241    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/583aca4d-1892-47a6-8f18-d4f1892ad8da-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:23.817367 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.817261    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:23.817367 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.817311    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "583aca4d-1892-47a6-8f18-d4f1892ad8da" (UID: "583aca4d-1892-47a6-8f18-d4f1892ad8da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:23.817367 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.817332    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "583aca4d-1892-47a6-8f18-d4f1892ad8da" (UID: "583aca4d-1892-47a6-8f18-d4f1892ad8da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:23.817680 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.817652    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-data" (OuterVolumeSpecName: "istio-data") pod "583aca4d-1892-47a6-8f18-d4f1892ad8da" (UID: "583aca4d-1892-47a6-8f18-d4f1892ad8da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:23.819017 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.818988    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "583aca4d-1892-47a6-8f18-d4f1892ad8da" (UID: "583aca4d-1892-47a6-8f18-d4f1892ad8da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:23.819141 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.819023    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-token" (OuterVolumeSpecName: "istio-token") pod "583aca4d-1892-47a6-8f18-d4f1892ad8da" (UID: "583aca4d-1892-47a6-8f18-d4f1892ad8da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:16:23.819141 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.819086    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-kube-api-access-nkxj2" (OuterVolumeSpecName: "kube-api-access-nkxj2") pod "583aca4d-1892-47a6-8f18-d4f1892ad8da" (UID: "583aca4d-1892-47a6-8f18-d4f1892ad8da"). InnerVolumeSpecName "kube-api-access-nkxj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:16:23.819141 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.819106    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "583aca4d-1892-47a6-8f18-d4f1892ad8da" (UID: "583aca4d-1892-47a6-8f18-d4f1892ad8da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 04:16:23.917936 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.917897    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:23.917936 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.917930    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:23.917936 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.917941    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:23.918181 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.917949    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:23.918181 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.917959    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:23.918181 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.917968    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/583aca4d-1892-47a6-8f18-d4f1892ad8da-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:23.918181 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:23.917977    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkxj2\" (UniqueName: \"kubernetes.io/projected/583aca4d-1892-47a6-8f18-d4f1892ad8da-kube-api-access-nkxj2\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:24.325124 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.325085    2559 generic.go:356] "Generic (PLEG): container finished" podID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerID="deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b" exitCode=0
Aug 04 04:16:24.325332 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.325132    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" event={"ID":"583aca4d-1892-47a6-8f18-d4f1892ad8da","Type":"ContainerDied","Data":"deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b"}
Aug 04 04:16:24.325332 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.325139    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"
Aug 04 04:16:24.325332 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.325197    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn" event={"ID":"583aca4d-1892-47a6-8f18-d4f1892ad8da","Type":"ContainerDied","Data":"65aee3c3795100b84ddac4c5a091a20ffccff74f073457620f0ff2da7a5d5721"}
Aug 04 04:16:24.325332 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.325220    2559 scope.go:117] "RemoveContainer" containerID="deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b"
Aug 04 04:16:24.335094 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.335066    2559 scope.go:117] "RemoveContainer" containerID="fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3"
Aug 04 04:16:24.345983 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.345948    2559 scope.go:117] "RemoveContainer" containerID="deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b"
Aug 04 04:16:24.346433 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:16:24.346413    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b\": container with ID starting with deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b not found: ID does not exist" containerID="deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b"
Aug 04 04:16:24.346498 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.346442    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b"} err="failed to get container status \"deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b\": rpc error: code = NotFound desc = could not find container \"deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b\": container with ID starting with deb8fdaad2287be2ffa499141eee5827d4f9d93b53deb6364d9e3bf508f88a4b not found: ID does not exist"
Aug 04 04:16:24.346498 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.346459    2559 scope.go:117] "RemoveContainer" containerID="fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3"
Aug 04 04:16:24.346744 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:16:24.346717    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3\": container with ID starting with fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3 not found: ID does not exist" containerID="fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3"
Aug 04 04:16:24.346789 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.346751    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3"} err="failed to get container status \"fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3\": rpc error: code = NotFound desc = could not find container \"fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3\": container with ID starting with fcf768252cc9523b9ef0709fa646dc934176b70de61645888fd601bfba5f4dc3 not found: ID does not exist"
Aug 04 04:16:24.350245 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.350217    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"]
Aug 04 04:16:24.353575 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:24.353552    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-b0089f-openshift-default-dd66649cd-ndpqn"]
Aug 04 04:16:25.238442 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:25.238407    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" path="/var/lib/kubelet/pods/583aca4d-1892-47a6-8f18-d4f1892ad8da/volumes"
Aug 04 04:16:29.551255 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:29.551217    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:29.578999 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:29.578962    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"]
Aug 04 04:16:29.579243 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:29.579214    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2" podUID="ed8d2b27-4b40-4a9e-9854-e6116715c53d" containerName="payload-processing" containerID="cri-o://9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192" gracePeriod=30
Aug 04 04:16:29.857275 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:29.857246    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:16:29.974157 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:29.974122    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8d2b27-4b40-4a9e-9854-e6116715c53d-plugins-config-volume\") pod \"ed8d2b27-4b40-4a9e-9854-e6116715c53d\" (UID: \"ed8d2b27-4b40-4a9e-9854-e6116715c53d\") "
Aug 04 04:16:29.974319 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:29.974193    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkmcw\" (UniqueName: \"kubernetes.io/projected/ed8d2b27-4b40-4a9e-9854-e6116715c53d-kube-api-access-vkmcw\") pod \"ed8d2b27-4b40-4a9e-9854-e6116715c53d\" (UID: \"ed8d2b27-4b40-4a9e-9854-e6116715c53d\") "
Aug 04 04:16:29.974492 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:29.974469    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed8d2b27-4b40-4a9e-9854-e6116715c53d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed8d2b27-4b40-4a9e-9854-e6116715c53d" (UID: "ed8d2b27-4b40-4a9e-9854-e6116715c53d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 04:16:29.976228 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:29.976205    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed8d2b27-4b40-4a9e-9854-e6116715c53d-kube-api-access-vkmcw" (OuterVolumeSpecName: "kube-api-access-vkmcw") pod "ed8d2b27-4b40-4a9e-9854-e6116715c53d" (UID: "ed8d2b27-4b40-4a9e-9854-e6116715c53d"). InnerVolumeSpecName "kube-api-access-vkmcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:16:30.075672 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:30.075625    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkmcw\" (UniqueName: \"kubernetes.io/projected/ed8d2b27-4b40-4a9e-9854-e6116715c53d-kube-api-access-vkmcw\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:30.075672 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:30.075661    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8d2b27-4b40-4a9e-9854-e6116715c53d-plugins-config-volume\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:30.358444 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:30.358403    2559 generic.go:356] "Generic (PLEG): container finished" podID="ed8d2b27-4b40-4a9e-9854-e6116715c53d" containerID="9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192" exitCode=0
Aug 04 04:16:30.358651 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:30.358453    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"
Aug 04 04:16:30.358651 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:30.358493    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2" event={"ID":"ed8d2b27-4b40-4a9e-9854-e6116715c53d","Type":"ContainerDied","Data":"9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192"}
Aug 04 04:16:30.358651 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:30.358580    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2" event={"ID":"ed8d2b27-4b40-4a9e-9854-e6116715c53d","Type":"ContainerDied","Data":"b28c4b81793d602c49a64e641d74c5aebb6e37a8732504e19ad5f073e098d8d2"}
Aug 04 04:16:30.358651 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:30.358604    2559 scope.go:117] "RemoveContainer" containerID="9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192"
Aug 04 04:16:30.367429 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:30.367407    2559 scope.go:117] "RemoveContainer" containerID="9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192"
Aug 04 04:16:30.367747 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:16:30.367731    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192\": container with ID starting with 9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192 not found: ID does not exist" containerID="9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192"
Aug 04 04:16:30.367821 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:30.367755    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192"} err="failed to get container status \"9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192\": rpc error: code = NotFound desc = could not find container \"9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192\": container with ID starting with 9365d1f6bf7511b95badab20d1b0dc4825b81fb7ce0c8c76c7f0e760cd12c192 not found: ID does not exist"
Aug 04 04:16:30.378912 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:30.378876    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"]
Aug 04 04:16:30.381024 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:30.380999    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-07c848-684dc5c578-c5dc2"]
Aug 04 04:16:31.034259 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:31.034225    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:31.238964 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:31.238932    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed8d2b27-4b40-4a9e-9854-e6116715c53d" path="/var/lib/kubelet/pods/ed8d2b27-4b40-4a9e-9854-e6116715c53d/volumes"
Aug 04 04:16:36.003286 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:36.003251    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:36.129180 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:36.129137    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:36.252584 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:36.252536    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"]
Aug 04 04:16:36.252797 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:36.252774    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy" containerID="cri-o://dfdd53fb0207683d3039fa57b81a1457bb55fbf67eae7be1d7466d4d9b0e8f87" gracePeriod=30
Aug 04 04:16:36.275397 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:36.275295    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:36.488881 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:36.488828    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:37.805442 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:37.805402    2559 patch_prober.go:28] interesting pod/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 04 04:16:37.805955 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:37.805485    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 04 04:16:41.422248 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.422213    2559 generic.go:356] "Generic (PLEG): container finished" podID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerID="dfdd53fb0207683d3039fa57b81a1457bb55fbf67eae7be1d7466d4d9b0e8f87" exitCode=0
Aug 04 04:16:41.422783 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.422283    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" event={"ID":"eca772f5-2b52-4600-9ba7-0bf3d94c528a","Type":"ContainerDied","Data":"dfdd53fb0207683d3039fa57b81a1457bb55fbf67eae7be1d7466d4d9b0e8f87"}
Aug 04 04:16:41.422783 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.422348    2559 scope.go:117] "RemoveContainer" containerID="65c6813035e7c2854fe7623eea21e42c81c1d2c5e416018e6375dcf8c9731afb"
Aug 04 04:16:41.513761 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.513685    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:16:41.584607 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.584530    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-token\") pod \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") "
Aug 04 04:16:41.584607 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.584611    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-podinfo\") pod \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") "
Aug 04 04:16:41.584843 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.584635    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-credential-socket\") pod \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") "
Aug 04 04:16:41.584843 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.584663    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-socket\") pod \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") "
Aug 04 04:16:41.584843 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.584690    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhl8m\" (UniqueName: \"kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-kube-api-access-jhl8m\") pod \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") "
Aug 04 04:16:41.584843 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.584737    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-envoy\") pod \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") "
Aug 04 04:16:41.584843 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.584762    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istiod-ca-cert\") pod \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") "
Aug 04 04:16:41.584843 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.584792    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-certs\") pod \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") "
Aug 04 04:16:41.584843 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.584842    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-data\") pod \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\" (UID: \"eca772f5-2b52-4600-9ba7-0bf3d94c528a\") "
Aug 04 04:16:41.585186 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.584977    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eca772f5-2b52-4600-9ba7-0bf3d94c528a" (UID: "eca772f5-2b52-4600-9ba7-0bf3d94c528a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:41.585186 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.584990    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eca772f5-2b52-4600-9ba7-0bf3d94c528a" (UID: "eca772f5-2b52-4600-9ba7-0bf3d94c528a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:41.585312 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.585188    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:41.585312 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.585210    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:41.585312 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.585223    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eca772f5-2b52-4600-9ba7-0bf3d94c528a" (UID: "eca772f5-2b52-4600-9ba7-0bf3d94c528a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:41.585312 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.585240    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eca772f5-2b52-4600-9ba7-0bf3d94c528a" (UID: "eca772f5-2b52-4600-9ba7-0bf3d94c528a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 04:16:41.586010 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.585980    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-data" (OuterVolumeSpecName: "istio-data") pod "eca772f5-2b52-4600-9ba7-0bf3d94c528a" (UID: "eca772f5-2b52-4600-9ba7-0bf3d94c528a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:41.587058 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.587027    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eca772f5-2b52-4600-9ba7-0bf3d94c528a" (UID: "eca772f5-2b52-4600-9ba7-0bf3d94c528a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 04:16:41.587058 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.587046    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-kube-api-access-jhl8m" (OuterVolumeSpecName: "kube-api-access-jhl8m") pod "eca772f5-2b52-4600-9ba7-0bf3d94c528a" (UID: "eca772f5-2b52-4600-9ba7-0bf3d94c528a"). InnerVolumeSpecName "kube-api-access-jhl8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:16:41.587199 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.587060    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-token" (OuterVolumeSpecName: "istio-token") pod "eca772f5-2b52-4600-9ba7-0bf3d94c528a" (UID: "eca772f5-2b52-4600-9ba7-0bf3d94c528a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:16:41.587199 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.587051    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eca772f5-2b52-4600-9ba7-0bf3d94c528a" (UID: "eca772f5-2b52-4600-9ba7-0bf3d94c528a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:41.685775 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.685728    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:41.685775 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.685771    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:41.685775 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.685780    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:41.685775 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.685788    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:41.685775 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.685797    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhl8m\" (UniqueName: \"kubernetes.io/projected/eca772f5-2b52-4600-9ba7-0bf3d94c528a-kube-api-access-jhl8m\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:41.686105 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.685806    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:41.686105 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:41.685814    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eca772f5-2b52-4600-9ba7-0bf3d94c528a-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:42.428754 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:42.428725    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"
Aug 04 04:16:42.429257 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:42.428720    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2" event={"ID":"eca772f5-2b52-4600-9ba7-0bf3d94c528a","Type":"ContainerDied","Data":"fe5adc10aa9fe30b37b62b562d195048b46b2d7483e789aa8d9143b99128ceda"}
Aug 04 04:16:42.429257 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:42.428872    2559 scope.go:117] "RemoveContainer" containerID="dfdd53fb0207683d3039fa57b81a1457bb55fbf67eae7be1d7466d4d9b0e8f87"
Aug 04 04:16:42.451988 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:42.451947    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"]
Aug 04 04:16:42.457361 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:42.457324    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-07c848-openshift-default-746f9df9b4-q26c2"]
Aug 04 04:16:43.238420 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:43.238377    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" path="/var/lib/kubelet/pods/eca772f5-2b52-4600-9ba7-0bf3d94c528a/volumes"
Aug 04 04:16:47.254954 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:47.254913    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:47.352617 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:47.352584    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:47.426506 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:47.426463    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:47.794232 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:47.794199    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:53.930422 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:53.930385    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:54.072945 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:54.072907    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:54.084905 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:54.084872    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"]
Aug 04 04:16:54.085183 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:54.085153    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy" containerID="cri-o://0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb" gracePeriod=30
Aug 04 04:16:54.138384 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:54.138348    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:54.327593 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:54.327537    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:16:54.767744 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:54.767706    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7885dbcc6c-gwc8b_64187948-6905-4ed9-a507-2c696c71ef90/authorino/0.log"
Aug 04 04:16:59.350772 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.350711    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:16:59.451583 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.451518    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-token\") pod \"6a1574b2-ce39-4e5f-bcee-9111a938a844\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") "
Aug 04 04:16:59.451583 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.451588    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-podinfo\") pod \"6a1574b2-ce39-4e5f-bcee-9111a938a844\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") "
Aug 04 04:16:59.451817 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.451744    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-credential-socket\") pod \"6a1574b2-ce39-4e5f-bcee-9111a938a844\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") "
Aug 04 04:16:59.451817 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.451794    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-envoy\") pod \"6a1574b2-ce39-4e5f-bcee-9111a938a844\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") "
Aug 04 04:16:59.451938 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.451826    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-certs\") pod \"6a1574b2-ce39-4e5f-bcee-9111a938a844\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") "
Aug 04 04:16:59.451938 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.451878    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-socket\") pod \"6a1574b2-ce39-4e5f-bcee-9111a938a844\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") "
Aug 04 04:16:59.451938 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.451913    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1574b2-ce39-4e5f-bcee-9111a938a844-istiod-ca-cert\") pod \"6a1574b2-ce39-4e5f-bcee-9111a938a844\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") "
Aug 04 04:16:59.452092 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.451949    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-data\") pod \"6a1574b2-ce39-4e5f-bcee-9111a938a844\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") "
Aug 04 04:16:59.452092 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.451991    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sgzx\" (UniqueName: \"kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-kube-api-access-8sgzx\") pod \"6a1574b2-ce39-4e5f-bcee-9111a938a844\" (UID: \"6a1574b2-ce39-4e5f-bcee-9111a938a844\") "
Aug 04 04:16:59.452092 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.452031    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a1574b2-ce39-4e5f-bcee-9111a938a844" (UID: "6a1574b2-ce39-4e5f-bcee-9111a938a844"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:59.452251 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.452095    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a1574b2-ce39-4e5f-bcee-9111a938a844" (UID: "6a1574b2-ce39-4e5f-bcee-9111a938a844"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:59.452427 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.452371    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a1574b2-ce39-4e5f-bcee-9111a938a844-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a1574b2-ce39-4e5f-bcee-9111a938a844" (UID: "6a1574b2-ce39-4e5f-bcee-9111a938a844"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 04:16:59.452427 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.452399    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-credential-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:59.452427 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.452421    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-socket\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:59.452770 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.452476    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a1574b2-ce39-4e5f-bcee-9111a938a844" (UID: "6a1574b2-ce39-4e5f-bcee-9111a938a844"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:59.452770 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.452737    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a1574b2-ce39-4e5f-bcee-9111a938a844" (UID: "6a1574b2-ce39-4e5f-bcee-9111a938a844"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:59.453969 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.453942    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a1574b2-ce39-4e5f-bcee-9111a938a844" (UID: "6a1574b2-ce39-4e5f-bcee-9111a938a844"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:16:59.454067 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.454014    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a1574b2-ce39-4e5f-bcee-9111a938a844" (UID: "6a1574b2-ce39-4e5f-bcee-9111a938a844"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 04:16:59.454190 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.454171    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a1574b2-ce39-4e5f-bcee-9111a938a844" (UID: "6a1574b2-ce39-4e5f-bcee-9111a938a844"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Aug 04 04:16:59.454316 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.454297    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-kube-api-access-8sgzx" (OuterVolumeSpecName: "kube-api-access-8sgzx") pod "6a1574b2-ce39-4e5f-bcee-9111a938a844" (UID: "6a1574b2-ce39-4e5f-bcee-9111a938a844"). InnerVolumeSpecName "kube-api-access-8sgzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 04:16:59.523677 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.523642    2559 generic.go:356] "Generic (PLEG): container finished" podID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerID="0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb" exitCode=0
Aug 04 04:16:59.523859 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.523690    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" event={"ID":"6a1574b2-ce39-4e5f-bcee-9111a938a844","Type":"ContainerDied","Data":"0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb"}
Aug 04 04:16:59.523859 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.523729    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg" event={"ID":"6a1574b2-ce39-4e5f-bcee-9111a938a844","Type":"ContainerDied","Data":"c8b64f481cb93cd6c06a66cdd4ab1446f0b1dc7c92c6bf8b752a6c4e51d54748"}
Aug 04 04:16:59.523859 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.523750    2559 scope.go:117] "RemoveContainer" containerID="0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb"
Aug 04 04:16:59.523859 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.523691    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"
Aug 04 04:16:59.533943 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.533921    2559 scope.go:117] "RemoveContainer" containerID="4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530"
Aug 04 04:16:59.542369 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.542348    2559 scope.go:117] "RemoveContainer" containerID="0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb"
Aug 04 04:16:59.542727 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:16:59.542704    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb\": container with ID starting with 0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb not found: ID does not exist" containerID="0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb"
Aug 04 04:16:59.542784 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.542737    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb"} err="failed to get container status \"0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb\": rpc error: code = NotFound desc = could not find container \"0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb\": container with ID starting with 0f81c55971d8d932c798ed2fcf481281e090357a2b9676a230ad5da947e6d2bb not found: ID does not exist"
Aug 04 04:16:59.542784 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.542768    2559 scope.go:117] "RemoveContainer" containerID="4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530"
Aug 04 04:16:59.543023 ip-10-0-134-110 kubenswrapper[2559]: E0804 04:16:59.543003    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530\": container with ID starting with 4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530 not found: ID does not exist" containerID="4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530"
Aug 04 04:16:59.543068 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.543029    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530"} err="failed to get container status \"4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530\": rpc error: code = NotFound desc = could not find container \"4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530\": container with ID starting with 4930e16ec220a69a9454101d1584ea598eba3c09814d799745b7155d87582530 not found: ID does not exist"
Aug 04 04:16:59.548416 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.548388    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"]
Aug 04 04:16:59.552166 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.552141    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-881c6f-openshift-default-69f5b95848-7hkzg"]
Aug 04 04:16:59.553758 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.553741    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-envoy\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:59.553809 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.553761    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-workload-certs\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:59.553809 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.553770    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1574b2-ce39-4e5f-bcee-9111a938a844-istiod-ca-cert\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:59.553809 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.553778    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-data\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:59.553809 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.553785    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sgzx\" (UniqueName: \"kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-kube-api-access-8sgzx\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:59.553809 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.553796    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-token\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:16:59.553809 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:16:59.553805    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1574b2-ce39-4e5f-bcee-9111a938a844-istio-podinfo\") on node \"ip-10-0-134-110.ec2.internal\" DevicePath \"\""
Aug 04 04:17:00.066600 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:00.066569    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xhgpl_327e7889-7bb5-4bac-8c54-929b09255d7f/manager/0.log"
Aug 04 04:17:00.185643 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:00.185599    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6z7g6_023e8e85-42fc-4303-9018-b02effcbfd18/manager/0.log"
Aug 04 04:17:00.434342 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:00.434253    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6r9tf_5260a6f3-0749-4ac6-8007-2ad027c6ee97/server/0.log"
Aug 04 04:17:00.697641 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:00.697565    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fb59c9fb6-hjw69_6d0d4c33-da26-4789-94bc-bbf9cd430b17/copy-manifests/0.log"
Aug 04 04:17:00.709439 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:00.709412    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fb59c9fb6-hjw69_6d0d4c33-da26-4789-94bc-bbf9cd430b17/manager/0.log"
Aug 04 04:17:01.237587 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:01.237534    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" path="/var/lib/kubelet/pods/6a1574b2-ce39-4e5f-bcee-9111a938a844/volumes"
Aug 04 04:17:01.417113 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:01.417029    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29763585-55m5l_bfe1439f-fe14-4695-8023-feacf4ff4cef/cleanup/0.log"
Aug 04 04:17:01.528298 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:01.528262    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29763600-ckcvk_df5487a0-7de8-43da-ae01-9d6ba493632d/cleanup/0.log"
Aug 04 04:17:03.032377 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:03.032344    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7885dbcc6c-gwc8b_64187948-6905-4ed9-a507-2c696c71ef90/authorino/0.log"
Aug 04 04:17:03.159507 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:03.159474    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-fg85w_4bb35e85-467d-448a-a0e2-e4a44cc7383d/manager/0.log"
Aug 04 04:17:03.268971 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:03.268944    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4fmw6_fafddda8-c6d9-4964-b15b-6f61a772fa1d/manager/0.log"
Aug 04 04:17:03.495436 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:03.495406    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zfsc4_79e97959-2959-4519-a3e3-d019d099070d/registry-server/0.log"
Aug 04 04:17:03.614932 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:03.614900    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2v7ls_0c243700-e14a-4859-8a81-b058e0bad3fb/manager/0.log"
Aug 04 04:17:03.859044 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:03.858961    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-r69f4_4ed94a51-6745-4d9b-8ecf-db3d7f8841f1/manager/0.log"
Aug 04 04:17:04.210600 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:04.210491    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr_86bee278-2fe4-430d-a7d9-017944b9590d/istio-proxy/0.log"
Aug 04 04:17:04.436956 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:04.436922    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69dbcc4dfb-8cjct_ab824707-d2bc-4302-af48-16e896a924f3/kube-auth-proxy/0.log"
Aug 04 04:17:04.647225 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:04.647141    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cbzlz_6f328e85-83a1-461f-913b-d9d77efb5bf9/istio-proxy/1.log"
Aug 04 04:17:04.762254 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:04.762218    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sg588_d8b40e05-241f-40a8-a72c-310f16ecbe23/payload-pre-processing/0.log"
Aug 04 04:17:05.340461 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:05.340430    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-6gzkg_ed2d64db-9822-49b4-9c97-052c175ad3df/main/0.log"
Aug 04 04:17:11.568554 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568513    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nsl8p/must-gather-d84zn"]
Aug 04 04:17:11.568961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568894    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy"
Aug 04 04:17:11.568961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568906    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy"
Aug 04 04:17:11.568961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568913    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy"
Aug 04 04:17:11.568961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568920    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy"
Aug 04 04:17:11.568961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568931    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy"
Aug 04 04:17:11.568961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568937    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy"
Aug 04 04:17:11.568961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568947    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed8d2b27-4b40-4a9e-9854-e6116715c53d" containerName="payload-processing"
Aug 04 04:17:11.568961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568952    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed8d2b27-4b40-4a9e-9854-e6116715c53d" containerName="payload-processing"
Aug 04 04:17:11.568961 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568963    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568970    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568976    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568982    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568987    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.568992    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569000    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b5a616e-9a87-4b93-b13c-755df2463220" containerName="main"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569005    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b5a616e-9a87-4b93-b13c-755df2463220" containerName="main"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569013    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569018    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569025    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7774e93f-75fa-4fc7-9dee-0a61a5945edd" containerName="payload-pre-processing"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569030    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7774e93f-75fa-4fc7-9dee-0a61a5945edd" containerName="payload-pre-processing"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569078    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569086    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a1574b2-ce39-4e5f-bcee-9111a938a844" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569091    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569097    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569103    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569108    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="583aca4d-1892-47a6-8f18-d4f1892ad8da" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569115    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc4f2a46-6e83-4f51-8d03-bbc3b90b6c3a" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569124    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7774e93f-75fa-4fc7-9dee-0a61a5945edd" containerName="payload-pre-processing"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569130    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b5a616e-9a87-4b93-b13c-755df2463220" containerName="main"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569139    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed8d2b27-4b40-4a9e-9854-e6116715c53d" containerName="payload-processing"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569205    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy"
Aug 04 04:17:11.569260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569212    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy"
Aug 04 04:17:11.569981 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.569278    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca772f5-2b52-4600-9ba7-0bf3d94c528a" containerName="istio-proxy"
Aug 04 04:17:11.572668 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.572641    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nsl8p/must-gather-d84zn"
Aug 04 04:17:11.575256 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.575229    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nsl8p\"/\"kube-root-ca.crt\""
Aug 04 04:17:11.575938 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.575915    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nsl8p\"/\"openshift-service-ca.crt\""
Aug 04 04:17:11.576074 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.575918    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nsl8p\"/\"default-dockercfg-c855t\""
Aug 04 04:17:11.579937 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.579907    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nsl8p/must-gather-d84zn"]
Aug 04 04:17:11.581887 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.581856    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:17:11.654590 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.654529    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log" limit=1024
Aug 04 04:17:11.665365 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.665330    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b491b6c9-a3f6-4fff-9e31-ab3512df40d0-must-gather-output\") pod \"must-gather-d84zn\" (UID: \"b491b6c9-a3f6-4fff-9e31-ab3512df40d0\") " pod="openshift-must-gather-nsl8p/must-gather-d84zn"
Aug 04 04:17:11.665565 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.665392    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2zvq\" (UniqueName: \"kubernetes.io/projected/b491b6c9-a3f6-4fff-9e31-ab3512df40d0-kube-api-access-q2zvq\") pod \"must-gather-d84zn\" (UID: \"b491b6c9-a3f6-4fff-9e31-ab3512df40d0\") " pod="openshift-must-gather-nsl8p/must-gather-d84zn"
Aug 04 04:17:11.766711 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.766676    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b491b6c9-a3f6-4fff-9e31-ab3512df40d0-must-gather-output\") pod \"must-gather-d84zn\" (UID: \"b491b6c9-a3f6-4fff-9e31-ab3512df40d0\") " pod="openshift-must-gather-nsl8p/must-gather-d84zn"
Aug 04 04:17:11.766885 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.766744    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2zvq\" (UniqueName: \"kubernetes.io/projected/b491b6c9-a3f6-4fff-9e31-ab3512df40d0-kube-api-access-q2zvq\") pod \"must-gather-d84zn\" (UID: \"b491b6c9-a3f6-4fff-9e31-ab3512df40d0\") " pod="openshift-must-gather-nsl8p/must-gather-d84zn"
Aug 04 04:17:11.767044 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.767026    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b491b6c9-a3f6-4fff-9e31-ab3512df40d0-must-gather-output\") pod \"must-gather-d84zn\" (UID: \"b491b6c9-a3f6-4fff-9e31-ab3512df40d0\") " pod="openshift-must-gather-nsl8p/must-gather-d84zn"
Aug 04 04:17:11.778938 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.778913    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2zvq\" (UniqueName: \"kubernetes.io/projected/b491b6c9-a3f6-4fff-9e31-ab3512df40d0-kube-api-access-q2zvq\") pod \"must-gather-d84zn\" (UID: \"b491b6c9-a3f6-4fff-9e31-ab3512df40d0\") " pod="openshift-must-gather-nsl8p/must-gather-d84zn"
Aug 04 04:17:11.884270 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:11.884145    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nsl8p/must-gather-d84zn"
Aug 04 04:17:12.025426 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:12.025386    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nsl8p/must-gather-d84zn"]
Aug 04 04:17:12.028245 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:17:12.028217    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb491b6c9_a3f6_4fff_9e31_ab3512df40d0.slice/crio-26a7ea7b53e35935ee89d705a000b3a00b3011fcf32c9ce7edd81a7438d844d0 WatchSource:0}: Error finding container 26a7ea7b53e35935ee89d705a000b3a00b3011fcf32c9ce7edd81a7438d844d0: Status 404 returned error can't find the container with id 26a7ea7b53e35935ee89d705a000b3a00b3011fcf32c9ce7edd81a7438d844d0
Aug 04 04:17:12.030694 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:12.030675    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 04:17:12.585797 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:12.585760    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nsl8p/must-gather-d84zn" event={"ID":"b491b6c9-a3f6-4fff-9e31-ab3512df40d0","Type":"ContainerStarted","Data":"26a7ea7b53e35935ee89d705a000b3a00b3011fcf32c9ce7edd81a7438d844d0"}
Aug 04 04:17:13.596567 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:13.595678    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nsl8p/must-gather-d84zn" event={"ID":"b491b6c9-a3f6-4fff-9e31-ab3512df40d0","Type":"ContainerStarted","Data":"cab8a1ce29590f8569686b90e9a1521d787f2ce1be8ad8d35e4651bd6b23ab0c"}
Aug 04 04:17:13.596567 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:13.595731    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nsl8p/must-gather-d84zn" event={"ID":"b491b6c9-a3f6-4fff-9e31-ab3512df40d0","Type":"ContainerStarted","Data":"74cb18fecd4190ca702b0c85961dcbc84f46b8b248497aaaf8a57335787c928f"}
Aug 04 04:17:13.613865 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:13.613803    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nsl8p/must-gather-d84zn" podStartSLOduration=1.9103322569999999 podStartE2EDuration="2.613778649s" totalImagesPullingTime="703.446392ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 04:17:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:17:13.61222621 +0000 UTC m=+4050.910366698" watchObservedRunningTime="2026-08-04 04:17:13.613778649 +0000 UTC m=+4050.911919138"
Aug 04 04:17:14.532136 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:14.532086    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xbn5d_550248ce-69c9-4f77-9e9e-f6f0a509f0c4/global-pull-secret-syncer/0.log"
Aug 04 04:17:14.589262 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:14.589216    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2svj7_48fcbb9a-7f25-411d-834f-00594bc5fb4f/konnectivity-agent/0.log"
Aug 04 04:17:14.720486 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:14.720458    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-110.ec2.internal_ce6294aa3b6a7973b0262394af529b36/haproxy/0.log"
Aug 04 04:17:20.283950 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:20.283913    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7885dbcc6c-gwc8b_64187948-6905-4ed9-a507-2c696c71ef90/authorino/0.log"
Aug 04 04:17:20.323704 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:20.323667    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-fg85w_4bb35e85-467d-448a-a0e2-e4a44cc7383d/manager/0.log"
Aug 04 04:17:20.356822 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:20.356765    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4fmw6_fafddda8-c6d9-4964-b15b-6f61a772fa1d/manager/0.log"
Aug 04 04:17:20.451925 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:20.451782    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zfsc4_79e97959-2959-4519-a3e3-d019d099070d/registry-server/0.log"
Aug 04 04:17:20.528369 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:20.528331    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2v7ls_0c243700-e14a-4859-8a81-b058e0bad3fb/manager/0.log"
Aug 04 04:17:20.644619 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:20.642952    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-r69f4_4ed94a51-6745-4d9b-8ecf-db3d7f8841f1/manager/0.log"
Aug 04 04:17:22.691725 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:22.691699    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-995qc_1f859f8e-2f0f-4072-8886-cf427010281b/node-exporter/0.log"
Aug 04 04:17:22.715876 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:22.715837    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-995qc_1f859f8e-2f0f-4072-8886-cf427010281b/kube-rbac-proxy/0.log"
Aug 04 04:17:22.741398 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:22.741363    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-995qc_1f859f8e-2f0f-4072-8886-cf427010281b/init-textfile/0.log"
Aug 04 04:17:23.306329 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:23.306298    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9479f9886-nhhhl_9decef81-74a7-4f6e-a2e6-befe2a57ef1e/telemeter-client/0.log"
Aug 04 04:17:23.328916 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:23.328888    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9479f9886-nhhhl_9decef81-74a7-4f6e-a2e6-befe2a57ef1e/reload/0.log"
Aug 04 04:17:23.350822 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:23.350789    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9479f9886-nhhhl_9decef81-74a7-4f6e-a2e6-befe2a57ef1e/kube-rbac-proxy/0.log"
Aug 04 04:17:25.474933 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.474894    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"]
Aug 04 04:17:25.484160 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.484125    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.489777 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.489747    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"]
Aug 04 04:17:25.519641 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.519601    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-sys\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.519833 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.519658    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz9fh\" (UniqueName: \"kubernetes.io/projected/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-kube-api-access-lz9fh\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.519833 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.519698    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-proc\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.519833 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.519755    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-lib-modules\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.519833 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.519821    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-podres\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.620930 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.620883    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-sys\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.620930 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.620922    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz9fh\" (UniqueName: \"kubernetes.io/projected/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-kube-api-access-lz9fh\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.621189 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.620942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-proc\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.621189 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.620979    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-lib-modules\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.621189 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.621011    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-sys\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.621189 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.621030    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-podres\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.621189 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.621075    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-proc\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.621189 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.621124    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-podres\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.621535 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.621208    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-lib-modules\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.631346 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.631315    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz9fh\" (UniqueName: \"kubernetes.io/projected/7fa1f6da-1859-4bec-a7ad-76e59ac05de0-kube-api-access-lz9fh\") pod \"perf-node-gather-daemonset-82w28\" (UID: \"7fa1f6da-1859-4bec-a7ad-76e59ac05de0\") " pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.800772 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.800732    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:25.972213 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:25.972168    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"]
Aug 04 04:17:25.974508 ip-10-0-134-110 kubenswrapper[2559]: W0804 04:17:25.974475    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7fa1f6da_1859_4bec_a7ad_76e59ac05de0.slice/crio-251020c41a3e46bc64cfa48b653f272446a4611b48138d28f1d689a23aaf9f53 WatchSource:0}: Error finding container 251020c41a3e46bc64cfa48b653f272446a4611b48138d28f1d689a23aaf9f53: Status 404 returned error can't find the container with id 251020c41a3e46bc64cfa48b653f272446a4611b48138d28f1d689a23aaf9f53
Aug 04 04:17:26.690435 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:26.690375    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28" event={"ID":"7fa1f6da-1859-4bec-a7ad-76e59ac05de0","Type":"ContainerStarted","Data":"85da2bf415c9d2fdc98a9e9c4e8da208e67f7fdd9870f2354f1b9db43ad92588"}
Aug 04 04:17:26.691005 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:26.690495    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28" event={"ID":"7fa1f6da-1859-4bec-a7ad-76e59ac05de0","Type":"ContainerStarted","Data":"251020c41a3e46bc64cfa48b653f272446a4611b48138d28f1d689a23aaf9f53"}
Aug 04 04:17:26.691005 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:26.690572    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:26.711260 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:26.711191    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28" podStartSLOduration=1.711157163 podStartE2EDuration="1.711157163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 04:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 04:17:26.708000373 +0000 UTC m=+4064.006140880" watchObservedRunningTime="2026-08-04 04:17:26.711157163 +0000 UTC m=+4064.009297653"
Aug 04 04:17:27.708537 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:27.708478    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-78fkp_a98e852a-83ee-40a7-8cc0-7243fcf6bafb/dns/0.log"
Aug 04 04:17:27.730722 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:27.730690    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-78fkp_a98e852a-83ee-40a7-8cc0-7243fcf6bafb/kube-rbac-proxy/0.log"
Aug 04 04:17:27.876302 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:27.876273    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f4mrm_6864c31c-d3e5-4224-beda-e85b0d20868d/dns-node-resolver/0.log"
Aug 04 04:17:28.434838 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:28.434803    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5dbf7b5fcf-j7pqz_977d3bf8-d024-4c65-85f6-5399b084433a/registry/0.log"
Aug 04 04:17:28.459239 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:28.459199    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9m9ks_7a2ea3c8-88ca-4802-987b-be5942899884/node-ca/0.log"
Aug 04 04:17:29.456792 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:29.456762    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr_86bee278-2fe4-430d-a7d9-017944b9590d/istio-proxy/0.log"
Aug 04 04:17:29.630570 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:29.630519    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69dbcc4dfb-8cjct_ab824707-d2bc-4302-af48-16e896a924f3/kube-auth-proxy/0.log"
Aug 04 04:17:29.684047 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:29.684014    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cbzlz_6f328e85-83a1-461f-913b-d9d77efb5bf9/istio-proxy/0.log"
Aug 04 04:17:29.717004 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:29.716924    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cbzlz_6f328e85-83a1-461f-913b-d9d77efb5bf9/istio-proxy/1.log"
Aug 04 04:17:29.748097 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:29.748068    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sg588_d8b40e05-241f-40a8-a72c-310f16ecbe23/payload-pre-processing/0.log"
Aug 04 04:17:30.627473 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:30.627446    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xqt4c_5eb1ab5d-b609-4822-baba-4357e434097f/serve-healthcheck-canary/0.log"
Aug 04 04:17:31.150858 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:31.150828    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k8plp_6da53b56-3fef-43d6-8a4b-7d7b590d0aa3/kube-rbac-proxy/0.log"
Aug 04 04:17:31.176613 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:31.176574    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k8plp_6da53b56-3fef-43d6-8a4b-7d7b590d0aa3/exporter/0.log"
Aug 04 04:17:31.198905 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:31.198877    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k8plp_6da53b56-3fef-43d6-8a4b-7d7b590d0aa3/extractor/0.log"
Aug 04 04:17:32.708363 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:32.708331    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nsl8p/perf-node-gather-daemonset-82w28"
Aug 04 04:17:33.505740 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:33.505712    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xhgpl_327e7889-7bb5-4bac-8c54-929b09255d7f/manager/0.log"
Aug 04 04:17:33.789504 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:33.789421    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6z7g6_023e8e85-42fc-4303-9018-b02effcbfd18/manager/0.log"
Aug 04 04:17:33.963454 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:33.963424    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6r9tf_5260a6f3-0749-4ac6-8007-2ad027c6ee97/server/0.log"
Aug 04 04:17:34.138880 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:34.138846    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fb59c9fb6-hjw69_6d0d4c33-da26-4789-94bc-bbf9cd430b17/manager/0.log"
Aug 04 04:17:34.155258 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:34.155233    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fb59c9fb6-hjw69_6d0d4c33-da26-4789-94bc-bbf9cd430b17/copy-manifests/0.log"
Aug 04 04:17:35.661628 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:35.661599    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr_86bee278-2fe4-430d-a7d9-017944b9590d/istio-proxy/0.log"
Aug 04 04:17:35.802058 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:35.802025    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-sjqdz_4537643e-dee5-49fa-a3b8-56157c00e58e/openshift-lws-operator/0.log"
Aug 04 04:17:35.814238 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:35.814158    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69dbcc4dfb-8cjct_ab824707-d2bc-4302-af48-16e896a924f3/kube-auth-proxy/0.log"
Aug 04 04:17:35.855477 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:35.855438    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cbzlz_6f328e85-83a1-461f-913b-d9d77efb5bf9/istio-proxy/0.log"
Aug 04 04:17:35.892236 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:35.892206    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cbzlz_6f328e85-83a1-461f-913b-d9d77efb5bf9/istio-proxy/1.log"
Aug 04 04:17:35.917849 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:35.917815    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sg588_d8b40e05-241f-40a8-a72c-310f16ecbe23/payload-pre-processing/0.log"
Aug 04 04:17:37.346467 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:37.346435    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2qplr_86bee278-2fe4-430d-a7d9-017944b9590d/istio-proxy/0.log"
Aug 04 04:17:37.496852 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:37.496828    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69dbcc4dfb-8cjct_ab824707-d2bc-4302-af48-16e896a924f3/kube-auth-proxy/0.log"
Aug 04 04:17:37.538945 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:37.538913    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cbzlz_6f328e85-83a1-461f-913b-d9d77efb5bf9/istio-proxy/0.log"
Aug 04 04:17:37.578648 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:37.578611    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-cbzlz_6f328e85-83a1-461f-913b-d9d77efb5bf9/istio-proxy/1.log"
Aug 04 04:17:37.602122 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:37.602086    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-sg588_d8b40e05-241f-40a8-a72c-310f16ecbe23/payload-pre-processing/0.log"
Aug 04 04:17:42.430394 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:42.430360    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dn58s_df20bbca-eb3a-44c3-be85-289103e0f04d/kube-multus-additional-cni-plugins/0.log"
Aug 04 04:17:42.453406 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:42.453378    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dn58s_df20bbca-eb3a-44c3-be85-289103e0f04d/egress-router-binary-copy/0.log"
Aug 04 04:17:42.478237 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:42.478214    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dn58s_df20bbca-eb3a-44c3-be85-289103e0f04d/cni-plugins/0.log"
Aug 04 04:17:42.500757 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:42.500730    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dn58s_df20bbca-eb3a-44c3-be85-289103e0f04d/bond-cni-plugin/0.log"
Aug 04 04:17:42.530369 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:42.530343    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dn58s_df20bbca-eb3a-44c3-be85-289103e0f04d/routeoverride-cni/0.log"
Aug 04 04:17:42.554494 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:42.554463    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dn58s_df20bbca-eb3a-44c3-be85-289103e0f04d/whereabouts-cni-bincopy/0.log"
Aug 04 04:17:42.576391 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:42.576367    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dn58s_df20bbca-eb3a-44c3-be85-289103e0f04d/whereabouts-cni/0.log"
Aug 04 04:17:42.804925 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:42.804896    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dj667_7dcf6319-3c75-4937-8f57-4c2735996721/kube-multus/0.log"
Aug 04 04:17:42.983278 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:42.983246    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xlcd8_57750f19-0d04-432e-94f1-d0d0fde883bf/network-metrics-daemon/0.log"
Aug 04 04:17:43.004352 ip-10-0-134-110 kubenswrapper[2559]: I0804 04:17:43.004317    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xlcd8_57750f19-0d04-432e-94f1-d0d0fde883bf/kube-rbac-proxy/0.log"