Jul 30 19:24:40.126392 ip-10-0-129-203 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 19:24:40.518622 ip-10-0-129-203 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 19:24:40.518622 ip-10-0-129-203 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 19:24:40.518622 ip-10-0-129-203 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 19:24:40.518622 ip-10-0-129-203 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 19:24:40.518622 ip-10-0-129-203 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 19:24:40.520281 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.520192    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 19:24:40.526167 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526148    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:40.526167 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526162    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:40.526167 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526166    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:40.526167 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526170    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:40.526167 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526173    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:40.526167 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526175    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526178    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526182    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526185    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526188    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526191    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526193    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526196    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526199    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526201    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526204    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526207    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526209    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526212    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526214    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526217    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526219    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526222    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526225    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526227    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:40.526380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526230    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526232    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526234    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526237    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526240    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526242    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526247    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526250    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526253    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526255    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526258    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526260    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526263    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526266    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526268    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526271    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526274    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526276    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526278    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:40.526864 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526281    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526283    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526286    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526288    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526291    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526293    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526296    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526299    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526301    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526304    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526306    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526308    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526311    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526313    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526317    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526320    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526324    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526328    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526331    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526334    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:40.527342 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526336    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526339    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526341    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526344    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526347    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526349    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526352    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526355    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526357    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526360    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526363    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526365    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526368    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526370    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526373    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526376    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526378    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526381    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526383    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526387    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:40.527817 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526389    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526392    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526394    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526398    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526400    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526403    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526407    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526413    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526417    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526420    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526422    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526425    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526428    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526430    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.526433    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527110    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527116    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527119    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527122    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:40.528311 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527125    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527129    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527131    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527134    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527137    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527139    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527142    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527145    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527147    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527150    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527152    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527155    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527157    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527160    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527163    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527166    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527168    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527171    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527173    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:40.528770 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527176    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527178    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527182    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527185    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527187    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527189    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527192    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527194    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527199    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527203    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527206    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527209    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527213    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527216    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527219    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527222    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527224    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527227    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527230    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:40.529250 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527233    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527236    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527238    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527241    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527243    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527247    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527249    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527252    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527255    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527257    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527260    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527262    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527265    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527267    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527270    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527273    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527275    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527278    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527280    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527283    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:40.529718 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527285    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527288    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527291    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527293    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527296    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527300    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527302    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527305    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527308    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527310    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527313    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527315    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527318    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527320    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527323    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527325    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527328    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527330    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527333    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527335    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:40.530218 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527338    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527342    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527346    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527350    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527354    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527357    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527361    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527363    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527366    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527369    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527371    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527374    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527376    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527379    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527381    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527384    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527387    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527807    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:40.530723 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527814    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527817    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527820    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527824    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527827    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527830    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527833    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527835    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527838    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527841    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527843    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527846    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527848    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527851    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527853    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527856    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527859    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527862    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527864    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527867    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:40.531181 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527870    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527873    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527876    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527878    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527881    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527883    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527886    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527888    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527891    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527893    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527896    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527898    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527901    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527904    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527906    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527909    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527911    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527914    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527917    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527919    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:40.531686 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527922    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527925    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527929    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527932    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527935    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527938    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527940    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527943    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527946    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527948    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527951    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527954    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527957    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527959    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527962    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527964    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527967    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527969    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527972    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:40.532186 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527974    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527977    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527980    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527982    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527985    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527987    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527990    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527993    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527995    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.527998    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528001    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528003    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528006    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528008    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528011    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528013    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528016    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528020    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528023    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528026    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:40.532656 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528028    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528030    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528033    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528036    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528039    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528042    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528044    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528047    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528050    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528052    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528054    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528057    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528061    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528063    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528081    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528085    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528088    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528090    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528093    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528152    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528158    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 19:24:40.533187 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528164    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528168    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528172    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528175    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528179    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528182    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528186    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528189    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528192    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528196    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528199    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528202    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528205    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528208    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528212    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528217    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528221    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528224    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528227    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528230    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528233    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528240    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528243    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 19:24:40.533697 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528246    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528249    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528260    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528265    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528268    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528271    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528274    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528278    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528281    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528284    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528287    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528290    2577 flags.go:64] FLAG: --enable-server="true"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528293    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528298    2577 flags.go:64] FLAG: --event-burst="100"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528301    2577 flags.go:64] FLAG: --event-qps="50"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528304    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528307    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528310    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528313    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528316    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528319    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528322    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528325    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528328    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528332    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 19:24:40.534319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528335    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528338    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528341    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528344    2577 flags.go:64] FLAG: --feature-gates=""
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528347    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528350    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528353    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528356    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528359    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528362    2577 flags.go:64] FLAG: --help="false"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528365    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-129-203.ec2.internal"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528368    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528371    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528373    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528377    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528380    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528383    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528385    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528388    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528391    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528394    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528397    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528400    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528402    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 19:24:40.534932 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528405    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528408    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528411    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528414    2577 flags.go:64] FLAG: --lock-file=""
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528416    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528419    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528422    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528428    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528432    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528434    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528437    2577 flags.go:64] FLAG: --logging-format="text"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528440    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528443    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528446    2577 flags.go:64] FLAG: --manifest-url=""
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528449    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528453    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528456    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528460    2577 flags.go:64] FLAG: --max-pods="110"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528463    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528466    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528468    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528476    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528479    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528482    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528485    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 19:24:40.535545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528491    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528494    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528497    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528500    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528503    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528508    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528511    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528514    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528517    2577 flags.go:64] FLAG: --port="10250"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528520    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528522    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bc4c06032ef97141"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528526    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528528    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528531    2577 flags.go:64] FLAG: --register-node="true"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528536    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528539    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528543    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528546    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528549    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528553    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528555    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528559    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528562    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528565    2577 flags.go:64] FLAG: --runonce="false"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528568    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 19:24:40.536182 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528571    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528574    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528577    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528580    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528583    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528586    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528589    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528592    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528595    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528597    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528601    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528603    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528606    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528609    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528615    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528617    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528620    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528624    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528626    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528629    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528632    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528636    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528639    2577 flags.go:64] FLAG: --v="2"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528643    2577 flags.go:64] FLAG: --version="false"
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528647    2577 flags.go:64] FLAG: --vmodule=""
Jul 30 19:24:40.536830 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528650    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.528653    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528758    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528764    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528767    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528770    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528774    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528777    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528780    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528783    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528785    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528788    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528791    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528793    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528796    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528799    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528802    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528804    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528807    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:40.537466 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528809    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528814    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528817    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528820    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528823    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528826    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528829    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528831    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528834    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528839    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528842    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528845    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528847    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528854    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528857    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528859    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528863    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528866    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528869    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:40.537966 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528871    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528874    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528876    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528879    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528881    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528884    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528886    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528889    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528892    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528894    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528897    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528899    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528902    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528906    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528909    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528912    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528915    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528918    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528921    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:40.538457 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528923    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528926    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528929    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528933    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528936    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528938    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528941    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528944    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528947    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528949    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528953    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528956    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528959    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528961    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528965    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528968    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528971    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528973    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528976    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528979    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:40.538928 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528981    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528984    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528986    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528989    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528991    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528994    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528997    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.528999    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529002    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529004    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529007    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529009    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529012    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529014    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529017    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529020    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529023    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529026    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529028    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529031    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:40.539488 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529033    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529036    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529040    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.529043    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.529565    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.535959    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.535975    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.535993    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.536002    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536055    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536060    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536063    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536082    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536085    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536087    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:40.539974 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536090    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536093    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536096    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536098    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536101    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536105    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536109    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536112    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536115    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536118    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536120    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536124    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536126    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536129    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536131    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536134    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536136    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536139    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536141    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:40.540388 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536144    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536146    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536148    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536151    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536154    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536157    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536160    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536162    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536165    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536167    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536170    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536172    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536175    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536177    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536180    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536182    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536185    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536187    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536190    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:40.540849 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536192    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536196    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536200    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536203    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536205    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536208    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536211    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536214    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536216    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536218    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536221    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536223    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536226    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536228    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536231    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536233    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536236    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536239    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536242    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:40.541332 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536244    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536247    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536250    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536258    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536261    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536264    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536267    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536269    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536272    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536275    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536277    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536280    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536282    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536284    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536287    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536289    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536292    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536294    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536297    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:40.541812 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536301    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536304    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536307    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536310    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536313    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536315    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536318    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536321    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536324    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536326    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536329    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536331    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536334    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536337    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536339    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536342    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:40.542389 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536344    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.536349    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536453    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536456    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536459    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536462    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536466    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536469    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536472    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536475    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536478    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536480    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536482    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536485    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536488    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:40.542777 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536490    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536493    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536496    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536500    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536503    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536506    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536508    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536510    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536513    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536516    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536518    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536520    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536523    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536525    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536528    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536530    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536533    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536536    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536538    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:40.543170 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536541    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536543    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536546    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536548    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536551    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536553    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536555    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536558    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536560    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536563    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536565    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536568    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536570    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536572    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536575    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536578    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536580    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536583    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536585    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:40.543776 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536587    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536590    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536592    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536595    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536597    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536599    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536602    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536605    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536607    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536610    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536613    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536615    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536617    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536620    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536622    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536626    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536629    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536632    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536635    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:40.544510 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536638    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536640    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536643    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536646    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536648    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536650    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536653    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536655    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536658    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536661    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536663    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536666    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536668    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536671    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536673    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536676    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536678    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536680    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536683    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536685    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:40.545094 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536688    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536690    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536693    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536695    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536698    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536701    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536703    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536705    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:40.536708    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.536712    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.536801    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.538628    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.540358    2577 server.go:1019] "Starting client certificate rotation"
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.540462    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 19:24:40.545575 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.540499    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 19:24:40.545911 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.542806    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 19:24:40.548303 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.548285    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 19:24:40.562152 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.562130    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 30 19:24:40.564336 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.564322    2577 log.go:25] "Validated CRI v1 image API"
Jul 30 19:24:40.565107 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.565090    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 19:24:40.567916 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.567896    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 19:24:40.568419 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.568395    2577 fs.go:135] Filesystem UUIDs: map[58a92950-de03-44a0-9d65-abe7b214aef6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8860abc6-3df9-42e8-b9f2-13270db849e6:/dev/nvme0n1p4]
Jul 30 19:24:40.568472 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.568419    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 19:24:40.573376 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.573278    2577 manager.go:216] Machine: {Timestamp:2026-07-30 19:24:40.571880964 +0000 UTC m=+0.332636213 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101988 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ff31245e69010d31cb320045182fd SystemUUID:ec2ff312-45e6-9010-d31c-b320045182fd BootID:97883764-975b-42ef-94e3-42fa5392a0ad Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f4:6b:b3:c3:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f4:6b:b3:c3:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:a0:d3:57:7b:f7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 19:24:40.573376 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.573375    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 19:24:40.573480 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.573440    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 19:24:40.576771 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.576743    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 19:24:40.576905 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.576774    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-203.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 19:24:40.577464 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.577453    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 19:24:40.577499 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.577465    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 19:24:40.577499 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.577473    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 19:24:40.578343 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.578333    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 19:24:40.579103 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.579093    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 19:24:40.582601 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.582590    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 19:24:40.582698 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.582689    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 19:24:40.582741 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.582719    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 19:24:40.582741 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.582728    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 19:24:40.582741 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.582741    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 19:24:40.582846 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.582750    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 30 19:24:40.582846 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.582761    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 19:24:40.583165 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.583143    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 19:24:40.583275 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.583174    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 19:24:40.585201 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.585181    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 19:24:40.586111 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.586094    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5k2jg"
Jul 30 19:24:40.587083 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.587050    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 19:24:40.587165 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.587158    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 19:24:40.588296 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588279    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 19:24:40.588296 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588298    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 19:24:40.588392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588309    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 19:24:40.588392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588334    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 19:24:40.588392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588344    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 19:24:40.588392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588354    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 19:24:40.588392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588363    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 19:24:40.588392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588369    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 19:24:40.588392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588375    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 19:24:40.588392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588381    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 19:24:40.588392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588390    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 19:24:40.588392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.588396    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 19:24:40.589193 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.589166    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 19:24:40.589228 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.589199    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 19:24:40.592737 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.592721    2577 server.go:1268] "Started kubelet"
Jul 30 19:24:40.592863 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.592814    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 19:24:40.592922 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.592885    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 19:24:40.593485 ip-10-0-129-203 systemd[1]: Started Kubernetes Kubelet.
Jul 30 19:24:40.593651 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.593108    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 19:24:40.593710 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.593676    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5k2jg"
Jul 30 19:24:40.593821 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.593806    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-203.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 19:24:40.593885 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.593856    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-203.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 19:24:40.593885 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.593849    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 19:24:40.594187 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.594177    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 19:24:40.595587 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.595569    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 19:24:40.598626 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.598604    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 19:24:40.599203 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.599186    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 19:24:40.599203 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.599192    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 19:24:40.599349 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.599281    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found"
Jul 30 19:24:40.599454 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.598413    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-203.ec2.internal.18c727bf78af688a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-203.ec2.internal,UID:ip-10-0-129-203.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-203.ec2.internal,},FirstTimestamp:2026-07-30 19:24:40.59268929 +0000 UTC m=+0.353444539,LastTimestamp:2026-07-30 19:24:40.59268929 +0000 UTC m=+0.353444539,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-203.ec2.internal,}"
Jul 30 19:24:40.599966 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.599952    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 19:24:40.600036 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.599990    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 19:24:40.600036 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.600018    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 19:24:40.600132 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.600096    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 19:24:40.600132 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.600109    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 19:24:40.600132 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.600118    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 19:24:40.600132 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.600109    2577 factory.go:55] Registering systemd factory
Jul 30 19:24:40.600268 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.600138    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 30 19:24:40.600380 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.600365    2577 factory.go:153] Registering CRI-O factory
Jul 30 19:24:40.600447 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.600384    2577 factory.go:223] Registration of the crio container factory successfully
Jul 30 19:24:40.600447 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.600407    2577 factory.go:103] Registering Raw factory
Jul 30 19:24:40.600447 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.600421    2577 manager.go:1196] Started watching for new ooms in manager
Jul 30 19:24:40.600861 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.600849    2577 manager.go:349] Starting recovery of all containers
Jul 30 19:24:40.607010 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.606987    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:40.609587 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.609560    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-203.ec2.internal\" not found" node="ip-10-0-129-203.ec2.internal"
Jul 30 19:24:40.611392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.611372    2577 manager.go:354] Recovery completed
Jul 30 19:24:40.617022 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.617009    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:40.620544 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.620530    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:40.620610 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.620561    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:40.620610 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.620576    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:40.621055 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.621039    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 19:24:40.621055 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.621053    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 19:24:40.621196 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.621088    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 19:24:40.621196 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.621095    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:40.622199 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.622183    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:40.622309 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.622206    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:40.622309 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.622220    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:40.622692 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.622679    2577 policy_none.go:49] "None policy: Start"
Jul 30 19:24:40.622744 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.622698    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 19:24:40.622744 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.622712    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 19:24:40.624054 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.624042    2577 policy_none.go:47] "Start"
Jul 30 19:24:40.667312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.667211    2577 manager.go:337] "Starting Device Plugin manager"
Jul 30 19:24:40.667393 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.667331    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 19:24:40.667393 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.667343    2577 server.go:87] "Starting device plugin registration server"
Jul 30 19:24:40.667568 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.667556    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 19:24:40.667626 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.667569    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 19:24:40.667693 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.667677    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 19:24:40.667775 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.667764    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 19:24:40.667775 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.667776    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 19:24:40.668228 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.668210    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 19:24:40.668296 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.668268    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-203.ec2.internal\" not found"
Jul 30 19:24:40.768250 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.768233    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:40.768970 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.768903    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:40.768970 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.768931    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:40.768970 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.768944    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:40.769168 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.768972    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-203.ec2.internal"
Jul 30 19:24:40.777004 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.776979    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 19:24:40.777966 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.777948    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-203.ec2.internal"
Jul 30 19:24:40.777966 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.777965    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-203.ec2.internal\": node \"ip-10-0-129-203.ec2.internal\" not found"
Jul 30 19:24:40.778124 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.777975    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 19:24:40.778124 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.778020    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 19:24:40.778124 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.778040    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 19:24:40.778124 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.778092    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 19:24:40.779575 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.779555    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:40.815697 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.815679    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found"
Jul 30 19:24:40.878753 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.878735    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal"]
Jul 30 19:24:40.878814 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.878795    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:40.880118 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.880105    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:40.880168 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.880130    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:40.880168 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.880139    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:40.881274 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.881262    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:40.881392 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.881379    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:40.881443 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.881409    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:40.881839 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.881817    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:40.881839 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.881828    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:40.881839 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.881836    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:40.881966 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.881843    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:40.881966 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.881849    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:40.881966 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.881852    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:40.882825 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.882812    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:40.882870 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.882833    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:40.883440 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.883426    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:40.883510 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.883449    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:40.883510 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.883458    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:40.901644 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.901629    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df1f3ece247108934ba885a4bf3aae23-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"df1f3ece247108934ba885a4bf3aae23\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:40.901713 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.901650    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df1f3ece247108934ba885a4bf3aae23-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"df1f3ece247108934ba885a4bf3aae23\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:40.901713 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:40.901666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c7684a39bc1278f7d06d66597af84743-config\") pod \"kube-apiserver-proxy-ip-10-0-129-203.ec2.internal\" (UID: \"c7684a39bc1278f7d06d66597af84743\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:40.903547 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.903532    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-203.ec2.internal\" not found" node="ip-10-0-129-203.ec2.internal"
Jul 30 19:24:40.907127 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.907111    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-203.ec2.internal\" not found" node="ip-10-0-129-203.ec2.internal"
Jul 30 19:24:40.915919 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:40.915907    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found"
Jul 30 19:24:41.001783 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.001764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df1f3ece247108934ba885a4bf3aae23-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"df1f3ece247108934ba885a4bf3aae23\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:41.001861 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.001789    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df1f3ece247108934ba885a4bf3aae23-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"df1f3ece247108934ba885a4bf3aae23\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:41.001861 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.001804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c7684a39bc1278f7d06d66597af84743-config\") pod \"kube-apiserver-proxy-ip-10-0-129-203.ec2.internal\" (UID: \"c7684a39bc1278f7d06d66597af84743\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:41.001861 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.001831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c7684a39bc1278f7d06d66597af84743-config\") pod \"kube-apiserver-proxy-ip-10-0-129-203.ec2.internal\" (UID: \"c7684a39bc1278f7d06d66597af84743\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:41.001861 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.001846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df1f3ece247108934ba885a4bf3aae23-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"df1f3ece247108934ba885a4bf3aae23\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:41.001861 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.001852    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df1f3ece247108934ba885a4bf3aae23-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"df1f3ece247108934ba885a4bf3aae23\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:41.016852 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:41.016834    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found"
Jul 30 19:24:41.117454 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:41.117413    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found"
Jul 30 19:24:41.205860 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.205844    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:41.210768 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.210752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:41.221250 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:41.221234    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found"
Jul 30 19:24:41.321629 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:41.321605    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found"
Jul 30 19:24:41.422059 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:41.422019    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found"
Jul 30 19:24:41.522497 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:41.522479    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found"
Jul 30 19:24:41.539631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.539615    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 19:24:41.539731 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.539717    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 19:24:41.539765 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.539751    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 19:24:41.598844 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.598819    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 19:19:40 +0000 UTC" deadline="2028-04-01 01:31:37.069894906 +0000 UTC"
Jul 30 19:24:41.598844 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.598841    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14646h6m55.471056659s"
Jul 30 19:24:41.603447 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.603433    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:41.699573 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.699525    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:41.709900 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.709883    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 19:24:41.711502 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.711479    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal"
Jul 30 19:24:41.718050 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.718034    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 19:24:41.732160 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:41.732125    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf1f3ece247108934ba885a4bf3aae23.slice/crio-85eabf270239f78e7c1b49d4f3ada9328071eeee410f4997ae53d748ab4dc63c WatchSource:0}: Error finding container 85eabf270239f78e7c1b49d4f3ada9328071eeee410f4997ae53d748ab4dc63c: Status 404 returned error can't find the container with id 85eabf270239f78e7c1b49d4f3ada9328071eeee410f4997ae53d748ab4dc63c
Jul 30 19:24:41.732395 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:41.732376    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7684a39bc1278f7d06d66597af84743.slice/crio-b4c6ca27a75aceb8879aa0d046ad36e15a769b895c33c45c2c5db74ae746ab44 WatchSource:0}: Error finding container b4c6ca27a75aceb8879aa0d046ad36e15a769b895c33c45c2c5db74ae746ab44: Status 404 returned error can't find the container with id b4c6ca27a75aceb8879aa0d046ad36e15a769b895c33c45c2c5db74ae746ab44
Jul 30 19:24:41.736904 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.736892    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 19:24:41.780404 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.780367    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" event={"ID":"c7684a39bc1278f7d06d66597af84743","Type":"ContainerStarted","Data":"b4c6ca27a75aceb8879aa0d046ad36e15a769b895c33c45c2c5db74ae746ab44"}
Jul 30 19:24:41.781278 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:41.781260    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" event={"ID":"df1f3ece247108934ba885a4bf3aae23","Type":"ContainerStarted","Data":"85eabf270239f78e7c1b49d4f3ada9328071eeee410f4997ae53d748ab4dc63c"}
Jul 30 19:24:42.193133 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.193053    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:42.429473 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.429445    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:42.546297 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.546220    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:42.584379 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.584355    2577 apiserver.go:52] "Watching apiserver"
Jul 30 19:24:42.591395 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.591372    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 19:24:42.591887 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.591865    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-4b287","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal","openshift-multus/multus-additional-cni-plugins-9b6xj","openshift-ovn-kubernetes/ovnkube-node-2r5sc","kube-system/global-pull-secret-syncer-7dmqn","openshift-image-registry/node-ca-7j96x","openshift-multus/multus-tgv2k","openshift-multus/network-metrics-daemon-26gjf","openshift-network-diagnostics/network-check-target-8ml9s","openshift-network-operator/iptables-alerter-bpbdk","kube-system/konnectivity-agent-p7ltn","kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg","openshift-cluster-node-tuning-operator/tuned-pcl9g"]
Jul 30 19:24:42.594420 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.593645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.595189 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.595051    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.596299 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.596272    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 19:24:42.596299 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.596287    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 19:24:42.596863 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.596528    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 19:24:42.596863 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.596636    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-szk5l\""
Jul 30 19:24:42.596863 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.596722    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 19:24:42.598446 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.598424    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 19:24:42.598446 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.598441    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 19:24:42.598595 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.598450    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-86lv5\""
Jul 30 19:24:42.598595 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.598503    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 19:24:42.598702 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.598627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.599882 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.599864    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.601202 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.600709    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 19:24:42.601202 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.600724    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nrmjb\""
Jul 30 19:24:42.601202 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.600955    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 19:24:42.601202 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.601166    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 19:24:42.602851 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.602690    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:42.602851 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:42.602754    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:24:42.604013 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.603178    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 19:24:42.604013 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.603411    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 19:24:42.604013 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.603459    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lj4fp\""
Jul 30 19:24:42.604013 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.603577    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 19:24:42.604013 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.603675    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 19:24:42.604013 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.603753    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 19:24:42.604013 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.603851    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 19:24:42.604013 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.603993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7j96x"
Jul 30 19:24:42.606831 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.605488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:42.606831 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:42.605555    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:24:42.606831 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.606311    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 19:24:42.606831 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.606537    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 19:24:42.607148 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.607129    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rkn2m\""
Jul 30 19:24:42.607355 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.607339    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 19:24:42.607544 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.607529    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4b287"
Jul 30 19:24:42.608867 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.608455    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.609512 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.609492    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 19:24:42.610000 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.609733    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xmj44\""
Jul 30 19:24:42.610252 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.610210    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 19:24:42.610347 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.610249    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bpbdk"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.611546    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dzj4w\""
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.611570    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.612347    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-var-lib-kubelet\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b945f77-954b-47f9-a8db-8c6c586dc346-cni-binary-copy\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-etc-openvswitch\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-multus-socket-dir-parent\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-var-lib-cni-bin\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-log-socket\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3fe769f8-bc2c-466c-be78-95ef7f74263c-dbus\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pg45\" (UniqueName: \"kubernetes.io/projected/0880f05e-770d-46dd-a957-08c121ec9fac-kube-api-access-6pg45\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7b945f77-954b-47f9-a8db-8c6c586dc346-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cslbb\" (UniqueName: \"kubernetes.io/projected/7b945f77-954b-47f9-a8db-8c6c586dc346-kube-api-access-cslbb\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-run-multus-certs\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-registration-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-slash\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.617406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-run-ovn-kubernetes\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-var-lib-openvswitch\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613799    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-os-release\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-os-release\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-system-cni-dir\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-system-cni-dir\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-cnibin\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-systemd-units\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.613980    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/01798510-b255-4582-adb9-866c4fbc4bb6-env-overrides\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7b945f77-954b-47f9-a8db-8c6c586dc346-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614093    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/01798510-b255-4582-adb9-866c4fbc4bb6-ovnkube-config\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-multus-conf-dir\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-device-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/01798510-b255-4582-adb9-866c4fbc4bb6-ovnkube-script-lib\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-var-lib-cni-multus\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.618255 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614250    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-run-netns\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614274    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3fe769f8-bc2c-466c-be78-95ef7f74263c-kubelet-config\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7khtf\" (UniqueName: \"kubernetes.io/projected/01798510-b255-4582-adb9-866c4fbc4bb6-kube-api-access-7khtf\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-run-netns\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614372    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-etc-kubernetes\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614401    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdn9b\" (UniqueName: \"kubernetes.io/projected/25ca9a86-bd6a-4728-adb6-954f15ebb254-kube-api-access-wdn9b\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614430    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-kubelet\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614461    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-run-openvswitch\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-run-ovn\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-multus-cni-dir\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-hostroot\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-socket-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614597    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-run-systemd\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614628    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-cni-netd\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614662    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/01798510-b255-4582-adb9-866c4fbc4bb6-ovn-node-metrics-cert\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-run-k8s-cni-cncf-io\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.618941 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614742    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-etc-selinux\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.619721 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-sys-fs\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.619721 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-cnibin\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.619721 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614823    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-node-log\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.619721 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.619721 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.614888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0880f05e-770d-46dd-a957-08c121ec9fac-cni-binary-copy\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.619721 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.615003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0880f05e-770d-46dd-a957-08c121ec9fac-multus-daemon-config\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.619721 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.615036    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.619721 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.615091    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-cni-bin\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.619721 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.615924    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:24:42.621896 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.620888    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 19:24:42.621896 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.621094    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-trzh8\""
Jul 30 19:24:42.621896 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.621201    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 19:24:42.621896 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.621276    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 19:24:42.621896 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.621418    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:24:42.624159 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.623149    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:42.624159 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.623328    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 19:24:42.624159 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.623616    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v7vb2\""
Jul 30 19:24:42.624159 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.623953    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 19:24:42.624830 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:42.624672    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:24:42.632749 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.632729    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-npx97"
Jul 30 19:24:42.643176 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.643138    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-npx97"
Jul 30 19:24:42.702022 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.701994    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7b945f77-954b-47f9-a8db-8c6c586dc346-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cslbb\" (UniqueName: \"kubernetes.io/projected/7b945f77-954b-47f9-a8db-8c6c586dc346-kube-api-access-cslbb\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-modprobe-d\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-run-multus-certs\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-registration-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-slash\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715409    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-run-ovn-kubernetes\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-lib-modules\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-var-lib-openvswitch\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-sys\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715496    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15de2c06-37ea-4122-9eaf-5fbe481c3127-konnectivity-ca\") pod \"konnectivity-agent-p7ltn\" (UID: \"15de2c06-37ea-4122-9eaf-5fbe481c3127\") " pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715513    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsx9g\" (UniqueName: \"kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g\") pod \"network-check-target-8ml9s\" (UID: \"6e6a374a-1f04-4351-bb56-8e329d65bc25\") " pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715537    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-os-release\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-os-release\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715592    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-system-cni-dir\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-system-cni-dir\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.717734 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-cnibin\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-systemd-units\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/01798510-b255-4582-adb9-866c4fbc4bb6-env-overrides\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-sysconfig\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87hg5\" (UniqueName: \"kubernetes.io/projected/f40f7c70-244b-4c1f-9d41-9485539a4ba2-kube-api-access-87hg5\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7b945f77-954b-47f9-a8db-8c6c586dc346-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/01798510-b255-4582-adb9-866c4fbc4bb6-ovnkube-config\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-var-lib-kubelet\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88cc7cdc-3575-4a0b-b5f3-740fb512673b-host-slash\") pod \"iptables-alerter-bpbdk\" (UID: \"88cc7cdc-3575-4a0b-b5f3-740fb512673b\") " pod="openshift-network-operator/iptables-alerter-bpbdk"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715855    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15de2c06-37ea-4122-9eaf-5fbe481c3127-agent-certs\") pod \"konnectivity-agent-p7ltn\" (UID: \"15de2c06-37ea-4122-9eaf-5fbe481c3127\") " pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/276fa5f2-21c9-42fc-88fc-6f7d3db559c9-hosts-file\") pod \"node-resolver-4b287\" (UID: \"276fa5f2-21c9-42fc-88fc-6f7d3db559c9\") " pod="openshift-dns/node-resolver-4b287"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1-serviceca\") pod \"node-ca-7j96x\" (UID: \"3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1\") " pod="openshift-image-registry/node-ca-7j96x"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-multus-conf-dir\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-device-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/01798510-b255-4582-adb9-866c4fbc4bb6-ovnkube-script-lib\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715967    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-systemd\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.718642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.715987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/276fa5f2-21c9-42fc-88fc-6f7d3db559c9-tmp-dir\") pod \"node-resolver-4b287\" (UID: \"276fa5f2-21c9-42fc-88fc-6f7d3db559c9\") " pod="openshift-dns/node-resolver-4b287"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-var-lib-cni-multus\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-run-netns\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3fe769f8-bc2c-466c-be78-95ef7f74263c-kubelet-config\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716114    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-sysctl-d\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-tuned\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88cc7cdc-3575-4a0b-b5f3-740fb512673b-iptables-alerter-script\") pod \"iptables-alerter-bpbdk\" (UID: \"88cc7cdc-3575-4a0b-b5f3-740fb512673b\") " pod="openshift-network-operator/iptables-alerter-bpbdk"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-systemd-units\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7khtf\" (UniqueName: \"kubernetes.io/projected/01798510-b255-4582-adb9-866c4fbc4bb6-kube-api-access-7khtf\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-multus-conf-dir\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:42.716329    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-registration-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716391    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-device-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:42.716439    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret podName:3fe769f8-bc2c-466c-be78-95ef7f74263c nodeName:}" failed. No retries permitted until 2026-07-30 19:24:43.216406279 +0000 UTC m=+2.977161535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret") pod "global-pull-secret-syncer-7dmqn" (UID: "3fe769f8-bc2c-466c-be78-95ef7f74263c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716476    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-run-multus-certs\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716678    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-os-release\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716759    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-run-netns\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.719470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716857    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/01798510-b255-4582-adb9-866c4fbc4bb6-env-overrides\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716864    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-slash\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716908    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-system-cni-dir\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.716935    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-os-release\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717010    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-cnibin\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717124    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-system-cni-dir\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nznjk\" (UniqueName: \"kubernetes.io/projected/88cc7cdc-3575-4a0b-b5f3-740fb512673b-kube-api-access-nznjk\") pod \"iptables-alerter-bpbdk\" (UID: \"88cc7cdc-3575-4a0b-b5f3-740fb512673b\") " pod="openshift-network-operator/iptables-alerter-bpbdk"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717191    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/01798510-b255-4582-adb9-866c4fbc4bb6-ovnkube-script-lib\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dktdj\" (UniqueName: \"kubernetes.io/projected/99fc519b-8e3c-4564-a65a-f81a314974f9-kube-api-access-dktdj\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fmwl\" (UniqueName: \"kubernetes.io/projected/3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1-kube-api-access-9fmwl\") pod \"node-ca-7j96x\" (UID: \"3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1\") " pod="openshift-image-registry/node-ca-7j96x"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-run-netns\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-etc-kubernetes\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717396    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7b945f77-954b-47f9-a8db-8c6c586dc346-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717396    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7b945f77-954b-47f9-a8db-8c6c586dc346-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdn9b\" (UniqueName: \"kubernetes.io/projected/25ca9a86-bd6a-4728-adb6-954f15ebb254-kube-api-access-wdn9b\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-kubelet\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-run-netns\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.720374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717494    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3fe769f8-bc2c-466c-be78-95ef7f74263c-kubelet-config\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717442    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-var-lib-cni-multus\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-run-openvswitch\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717544    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-kubelet\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-run-ovn\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1-host\") pod \"node-ca-7j96x\" (UID: \"3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1\") " pod="openshift-image-registry/node-ca-7j96x"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717612    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-var-lib-openvswitch\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-multus-cni-dir\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717657    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-hostroot\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-kubernetes\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-run-ovn\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717690    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-run-openvswitch\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-run\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-hostroot\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-socket-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717760    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-run-ovn-kubernetes\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-run-systemd\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-cni-netd\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717881    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-multus-cni-dir\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/01798510-b255-4582-adb9-866c4fbc4bb6-ovn-node-metrics-cert\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717909    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-socket-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-run-systemd\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-run-k8s-cni-cncf-io\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-cni-netd\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-etc-kubernetes\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.717977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-etc-selinux\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-sys-fs\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718015    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/01798510-b255-4582-adb9-866c4fbc4bb6-ovnkube-config\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718019    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-run-k8s-cni-cncf-io\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-cnibin\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718084    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-node-log\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718088    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-sys-fs\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-cnibin\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-etc-selinux\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.721850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718135    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-node-log\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718159    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-sysctl-conf\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0880f05e-770d-46dd-a957-08c121ec9fac-cni-binary-copy\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0880f05e-770d-46dd-a957-08c121ec9fac-multus-daemon-config\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718301    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-cni-bin\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718341    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7j94\" (UniqueName: \"kubernetes.io/projected/276fa5f2-21c9-42fc-88fc-6f7d3db559c9-kube-api-access-f7j94\") pod \"node-resolver-4b287\" (UID: \"276fa5f2-21c9-42fc-88fc-6f7d3db559c9\") " pod="openshift-dns/node-resolver-4b287"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-var-lib-kubelet\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b945f77-954b-47f9-a8db-8c6c586dc346-cni-binary-copy\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718478    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-etc-openvswitch\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718484    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-host-cni-bin\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-host\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-multus-socket-dir-parent\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-var-lib-cni-bin\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.722631 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-log-socket\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b945f77-954b-47f9-a8db-8c6c586dc346-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3fe769f8-bc2c-466c-be78-95ef7f74263c-dbus\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f40f7c70-244b-4c1f-9d41-9485539a4ba2-tmp\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718796    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-etc-openvswitch\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pg45\" (UniqueName: \"kubernetes.io/projected/0880f05e-770d-46dd-a957-08c121ec9fac-kube-api-access-6pg45\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718912    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-var-lib-cni-bin\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718925    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-multus-socket-dir-parent\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718937    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0880f05e-770d-46dd-a957-08c121ec9fac-cni-binary-copy\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718967    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0880f05e-770d-46dd-a957-08c121ec9fac-multus-daemon-config\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.718981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01798510-b255-4582-adb9-866c4fbc4bb6-log-socket\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.719026    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0880f05e-770d-46dd-a957-08c121ec9fac-host-var-lib-kubelet\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.719086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25ca9a86-bd6a-4728-adb6-954f15ebb254-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.719113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3fe769f8-bc2c-466c-be78-95ef7f74263c-dbus\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:42.723312 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.719461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b945f77-954b-47f9-a8db-8c6c586dc346-cni-binary-copy\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.737652 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.736888    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/01798510-b255-4582-adb9-866c4fbc4bb6-ovn-node-metrics-cert\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.737652 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.736946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pg45\" (UniqueName: \"kubernetes.io/projected/0880f05e-770d-46dd-a957-08c121ec9fac-kube-api-access-6pg45\") pod \"multus-tgv2k\" (UID: \"0880f05e-770d-46dd-a957-08c121ec9fac\") " pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.737652 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.736958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7khtf\" (UniqueName: \"kubernetes.io/projected/01798510-b255-4582-adb9-866c4fbc4bb6-kube-api-access-7khtf\") pod \"ovnkube-node-2r5sc\" (UID: \"01798510-b255-4582-adb9-866c4fbc4bb6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.737652 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.737089    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdn9b\" (UniqueName: \"kubernetes.io/projected/25ca9a86-bd6a-4728-adb6-954f15ebb254-kube-api-access-wdn9b\") pod \"aws-ebs-csi-driver-node-g4nvg\" (UID: \"25ca9a86-bd6a-4728-adb6-954f15ebb254\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.737652 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.737094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cslbb\" (UniqueName: \"kubernetes.io/projected/7b945f77-954b-47f9-a8db-8c6c586dc346-kube-api-access-cslbb\") pod \"multus-additional-cni-plugins-9b6xj\" (UID: \"7b945f77-954b-47f9-a8db-8c6c586dc346\") " pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.819786 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-lib-modules\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.819786 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819758    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-sys\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.819786 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15de2c06-37ea-4122-9eaf-5fbe481c3127-konnectivity-ca\") pod \"konnectivity-agent-p7ltn\" (UID: \"15de2c06-37ea-4122-9eaf-5fbe481c3127\") " pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:24:42.820001 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsx9g\" (UniqueName: \"kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g\") pod \"network-check-target-8ml9s\" (UID: \"6e6a374a-1f04-4351-bb56-8e329d65bc25\") " pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:42.820001 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819861    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-sys\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820001 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-sysconfig\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820001 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-lib-modules\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820001 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819912    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87hg5\" (UniqueName: \"kubernetes.io/projected/f40f7c70-244b-4c1f-9d41-9485539a4ba2-kube-api-access-87hg5\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820001 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819922    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-sysconfig\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820001 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-var-lib-kubelet\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820001 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819964    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88cc7cdc-3575-4a0b-b5f3-740fb512673b-host-slash\") pod \"iptables-alerter-bpbdk\" (UID: \"88cc7cdc-3575-4a0b-b5f3-740fb512673b\") " pod="openshift-network-operator/iptables-alerter-bpbdk"
Jul 30 19:24:42.820001 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.819989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15de2c06-37ea-4122-9eaf-5fbe481c3127-agent-certs\") pod \"konnectivity-agent-p7ltn\" (UID: \"15de2c06-37ea-4122-9eaf-5fbe481c3127\") " pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/276fa5f2-21c9-42fc-88fc-6f7d3db559c9-hosts-file\") pod \"node-resolver-4b287\" (UID: \"276fa5f2-21c9-42fc-88fc-6f7d3db559c9\") " pod="openshift-dns/node-resolver-4b287"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1-serviceca\") pod \"node-ca-7j96x\" (UID: \"3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1\") " pod="openshift-image-registry/node-ca-7j96x"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820059    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-systemd\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-var-lib-kubelet\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820103    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/276fa5f2-21c9-42fc-88fc-6f7d3db559c9-tmp-dir\") pod \"node-resolver-4b287\" (UID: \"276fa5f2-21c9-42fc-88fc-6f7d3db559c9\") " pod="openshift-dns/node-resolver-4b287"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820120    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/276fa5f2-21c9-42fc-88fc-6f7d3db559c9-hosts-file\") pod \"node-resolver-4b287\" (UID: \"276fa5f2-21c9-42fc-88fc-6f7d3db559c9\") " pod="openshift-dns/node-resolver-4b287"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820129    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-sysctl-d\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820135    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88cc7cdc-3575-4a0b-b5f3-740fb512673b-host-slash\") pod \"iptables-alerter-bpbdk\" (UID: \"88cc7cdc-3575-4a0b-b5f3-740fb512673b\") " pod="openshift-network-operator/iptables-alerter-bpbdk"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-tuned\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88cc7cdc-3575-4a0b-b5f3-740fb512673b-iptables-alerter-script\") pod \"iptables-alerter-bpbdk\" (UID: \"88cc7cdc-3575-4a0b-b5f3-740fb512673b\") " pod="openshift-network-operator/iptables-alerter-bpbdk"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820191    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-systemd\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nznjk\" (UniqueName: \"kubernetes.io/projected/88cc7cdc-3575-4a0b-b5f3-740fb512673b-kube-api-access-nznjk\") pod \"iptables-alerter-bpbdk\" (UID: \"88cc7cdc-3575-4a0b-b5f3-740fb512673b\") " pod="openshift-network-operator/iptables-alerter-bpbdk"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820236    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dktdj\" (UniqueName: \"kubernetes.io/projected/99fc519b-8e3c-4564-a65a-f81a314974f9-kube-api-access-dktdj\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-sysctl-d\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fmwl\" (UniqueName: \"kubernetes.io/projected/3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1-kube-api-access-9fmwl\") pod \"node-ca-7j96x\" (UID: \"3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1\") " pod="openshift-image-registry/node-ca-7j96x"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1-host\") pod \"node-ca-7j96x\" (UID: \"3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1\") " pod="openshift-image-registry/node-ca-7j96x"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-kubernetes\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.820434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-run\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-sysctl-conf\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820383    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15de2c06-37ea-4122-9eaf-5fbe481c3127-konnectivity-ca\") pod \"konnectivity-agent-p7ltn\" (UID: \"15de2c06-37ea-4122-9eaf-5fbe481c3127\") " pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7j94\" (UniqueName: \"kubernetes.io/projected/276fa5f2-21c9-42fc-88fc-6f7d3db559c9-kube-api-access-f7j94\") pod \"node-resolver-4b287\" (UID: \"276fa5f2-21c9-42fc-88fc-6f7d3db559c9\") " pod="openshift-dns/node-resolver-4b287"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820430    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/276fa5f2-21c9-42fc-88fc-6f7d3db559c9-tmp-dir\") pod \"node-resolver-4b287\" (UID: \"276fa5f2-21c9-42fc-88fc-6f7d3db559c9\") " pod="openshift-dns/node-resolver-4b287"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820475    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-run\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820559    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-sysctl-conf\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:42.820634    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820635    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1-host\") pod \"node-ca-7j96x\" (UID: \"3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1\") " pod="openshift-image-registry/node-ca-7j96x"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:42.820685    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs podName:99fc519b-8e3c-4564-a65a-f81a314974f9 nodeName:}" failed. No retries permitted until 2026-07-30 19:24:43.320668337 +0000 UTC m=+3.081423591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs") pod "network-metrics-daemon-26gjf" (UID: "99fc519b-8e3c-4564-a65a-f81a314974f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-kubernetes\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-host\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f40f7c70-244b-4c1f-9d41-9485539a4ba2-tmp\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820749    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-host\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-modprobe-d\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820851    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/88cc7cdc-3575-4a0b-b5f3-740fb512673b-iptables-alerter-script\") pod \"iptables-alerter-bpbdk\" (UID: \"88cc7cdc-3575-4a0b-b5f3-740fb512673b\") " pod="openshift-network-operator/iptables-alerter-bpbdk"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.820902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-modprobe-d\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.821174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.821059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1-serviceca\") pod \"node-ca-7j96x\" (UID: \"3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1\") " pod="openshift-image-registry/node-ca-7j96x"
Jul 30 19:24:42.822910 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.822890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f40f7c70-244b-4c1f-9d41-9485539a4ba2-etc-tuned\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.823268 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.823246    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15de2c06-37ea-4122-9eaf-5fbe481c3127-agent-certs\") pod \"konnectivity-agent-p7ltn\" (UID: \"15de2c06-37ea-4122-9eaf-5fbe481c3127\") " pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:24:42.823480 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.823461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f40f7c70-244b-4c1f-9d41-9485539a4ba2-tmp\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.825181 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:42.825154    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:42.825181 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:42.825178    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:42.825324 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:42.825193    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lsx9g for pod openshift-network-diagnostics/network-check-target-8ml9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:42.825324 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:42.825262    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g podName:6e6a374a-1f04-4351-bb56-8e329d65bc25 nodeName:}" failed. No retries permitted until 2026-07-30 19:24:43.32524495 +0000 UTC m=+3.086000192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lsx9g" (UniqueName: "kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g") pod "network-check-target-8ml9s" (UID: "6e6a374a-1f04-4351-bb56-8e329d65bc25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:42.827152 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.827130    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87hg5\" (UniqueName: \"kubernetes.io/projected/f40f7c70-244b-4c1f-9d41-9485539a4ba2-kube-api-access-87hg5\") pod \"tuned-pcl9g\" (UID: \"f40f7c70-244b-4c1f-9d41-9485539a4ba2\") " pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.830669 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.830634    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fmwl\" (UniqueName: \"kubernetes.io/projected/3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1-kube-api-access-9fmwl\") pod \"node-ca-7j96x\" (UID: \"3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1\") " pod="openshift-image-registry/node-ca-7j96x"
Jul 30 19:24:42.831038 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.831017    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7j94\" (UniqueName: \"kubernetes.io/projected/276fa5f2-21c9-42fc-88fc-6f7d3db559c9-kube-api-access-f7j94\") pod \"node-resolver-4b287\" (UID: \"276fa5f2-21c9-42fc-88fc-6f7d3db559c9\") " pod="openshift-dns/node-resolver-4b287"
Jul 30 19:24:42.831155 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.831023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nznjk\" (UniqueName: \"kubernetes.io/projected/88cc7cdc-3575-4a0b-b5f3-740fb512673b-kube-api-access-nznjk\") pod \"iptables-alerter-bpbdk\" (UID: \"88cc7cdc-3575-4a0b-b5f3-740fb512673b\") " pod="openshift-network-operator/iptables-alerter-bpbdk"
Jul 30 19:24:42.831231 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.831211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dktdj\" (UniqueName: \"kubernetes.io/projected/99fc519b-8e3c-4564-a65a-f81a314974f9-kube-api-access-dktdj\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:42.912147 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.912120    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tgv2k"
Jul 30 19:24:42.935915 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.935888    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg"
Jul 30 19:24:42.944432 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.944408    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9b6xj"
Jul 30 19:24:42.952111 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.952089    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:24:42.959924 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.959907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7j96x"
Jul 30 19:24:42.966507 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.966488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4b287"
Jul 30 19:24:42.974134 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.974118    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pcl9g"
Jul 30 19:24:42.982712 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.982692    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bpbdk"
Jul 30 19:24:42.988306 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:42.988293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:24:43.224679 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.224607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:43.224797 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:43.224724    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:43.224797 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:43.224772    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret podName:3fe769f8-bc2c-466c-be78-95ef7f74263c nodeName:}" failed. No retries permitted until 2026-07-30 19:24:44.224758844 +0000 UTC m=+3.985514081 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret") pod "global-pull-secret-syncer-7dmqn" (UID: "3fe769f8-bc2c-466c-be78-95ef7f74263c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:43.325305 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.325270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:43.325452 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:43.325378    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:43.325452 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:43.325431    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs podName:99fc519b-8e3c-4564-a65a-f81a314974f9 nodeName:}" failed. No retries permitted until 2026-07-30 19:24:44.325418193 +0000 UTC m=+4.086173430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs") pod "network-metrics-daemon-26gjf" (UID: "99fc519b-8e3c-4564-a65a-f81a314974f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:43.425859 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.425826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsx9g\" (UniqueName: \"kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g\") pod \"network-check-target-8ml9s\" (UID: \"6e6a374a-1f04-4351-bb56-8e329d65bc25\") " pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:43.426017 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:43.425984    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:43.426017 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:43.426005    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:43.426017 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:43.426015    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lsx9g for pod openshift-network-diagnostics/network-check-target-8ml9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:43.426162 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:43.426080    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g podName:6e6a374a-1f04-4351-bb56-8e329d65bc25 nodeName:}" failed. No retries permitted until 2026-07-30 19:24:44.426050835 +0000 UTC m=+4.186806071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsx9g" (UniqueName: "kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g") pod "network-check-target-8ml9s" (UID: "6e6a374a-1f04-4351-bb56-8e329d65bc25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:43.553145 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:43.553112    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15de2c06_37ea_4122_9eaf_5fbe481c3127.slice/crio-ed8b41316f95587f20fd7f20c86fbffa58c0d16d4c378a5f1a5a4b1a48175c4c WatchSource:0}: Error finding container ed8b41316f95587f20fd7f20c86fbffa58c0d16d4c378a5f1a5a4b1a48175c4c: Status 404 returned error can't find the container with id ed8b41316f95587f20fd7f20c86fbffa58c0d16d4c378a5f1a5a4b1a48175c4c
Jul 30 19:24:43.554589 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:43.554209    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25ca9a86_bd6a_4728_adb6_954f15ebb254.slice/crio-82ed7eccd3405b59e0847308f9aae3dcbb851fd4a2f31368e1229482ab076090 WatchSource:0}: Error finding container 82ed7eccd3405b59e0847308f9aae3dcbb851fd4a2f31368e1229482ab076090: Status 404 returned error can't find the container with id 82ed7eccd3405b59e0847308f9aae3dcbb851fd4a2f31368e1229482ab076090
Jul 30 19:24:43.557262 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:43.557242    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0880f05e_770d_46dd_a957_08c121ec9fac.slice/crio-105d71a49625803bd76221ed5446117fe44558d42cea568e2d66ace3ac24d0b9 WatchSource:0}: Error finding container 105d71a49625803bd76221ed5446117fe44558d42cea568e2d66ace3ac24d0b9: Status 404 returned error can't find the container with id 105d71a49625803bd76221ed5446117fe44558d42cea568e2d66ace3ac24d0b9
Jul 30 19:24:43.557977 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:43.557957    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf40f7c70_244b_4c1f_9d41_9485539a4ba2.slice/crio-f9bc8ad651954465e2e1600e646aa3511cdb07d970fae0cd3ce4c40d2b88481f WatchSource:0}: Error finding container f9bc8ad651954465e2e1600e646aa3511cdb07d970fae0cd3ce4c40d2b88481f: Status 404 returned error can't find the container with id f9bc8ad651954465e2e1600e646aa3511cdb07d970fae0cd3ce4c40d2b88481f
Jul 30 19:24:43.559223 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:43.559201    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88cc7cdc_3575_4a0b_b5f3_740fb512673b.slice/crio-4a6e46687cc643ec9e4645e720ec515b8f2bd3f9d2f2187709f9e5610efbd0c4 WatchSource:0}: Error finding container 4a6e46687cc643ec9e4645e720ec515b8f2bd3f9d2f2187709f9e5610efbd0c4: Status 404 returned error can't find the container with id 4a6e46687cc643ec9e4645e720ec515b8f2bd3f9d2f2187709f9e5610efbd0c4
Jul 30 19:24:43.559766 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:43.559727    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b945f77_954b_47f9_a8db_8c6c586dc346.slice/crio-954e086df7362c83ba9ab38310de54b3f7c5b1bc8ffe5442ee0148ca364e7dd1 WatchSource:0}: Error finding container 954e086df7362c83ba9ab38310de54b3f7c5b1bc8ffe5442ee0148ca364e7dd1: Status 404 returned error can't find the container with id 954e086df7362c83ba9ab38310de54b3f7c5b1bc8ffe5442ee0148ca364e7dd1
Jul 30 19:24:43.561086 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:43.560802    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a3a6e9b_c842_4a6d_a79f_49efa6cf02a1.slice/crio-cc4ca4d8806f9b2ff81d30c3c62546096edecb3d76835376e44d13c7c32a8d9e WatchSource:0}: Error finding container cc4ca4d8806f9b2ff81d30c3c62546096edecb3d76835376e44d13c7c32a8d9e: Status 404 returned error can't find the container with id cc4ca4d8806f9b2ff81d30c3c62546096edecb3d76835376e44d13c7c32a8d9e
Jul 30 19:24:43.561670 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:24:43.561649    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01798510_b255_4582_adb9_866c4fbc4bb6.slice/crio-890f9b70b17dd0eb22bc71d8f9e0292cd539e769bf70c6dbe54351153d341b80 WatchSource:0}: Error finding container 890f9b70b17dd0eb22bc71d8f9e0292cd539e769bf70c6dbe54351153d341b80: Status 404 returned error can't find the container with id 890f9b70b17dd0eb22bc71d8f9e0292cd539e769bf70c6dbe54351153d341b80
Jul 30 19:24:43.645164 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.644954    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 19:19:42 +0000 UTC" deadline="2028-05-12 16:12:15.714443178 +0000 UTC"
Jul 30 19:24:43.645164 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.645164    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15644h47m32.06928472s"
Jul 30 19:24:43.778845 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.778778    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:43.778946 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:43.778928    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:24:43.786806 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.786781    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pcl9g" event={"ID":"f40f7c70-244b-4c1f-9d41-9485539a4ba2","Type":"ContainerStarted","Data":"f9bc8ad651954465e2e1600e646aa3511cdb07d970fae0cd3ce4c40d2b88481f"}
Jul 30 19:24:43.787922 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.787898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tgv2k" event={"ID":"0880f05e-770d-46dd-a957-08c121ec9fac","Type":"ContainerStarted","Data":"105d71a49625803bd76221ed5446117fe44558d42cea568e2d66ace3ac24d0b9"}
Jul 30 19:24:43.789022 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.789001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg" event={"ID":"25ca9a86-bd6a-4728-adb6-954f15ebb254","Type":"ContainerStarted","Data":"82ed7eccd3405b59e0847308f9aae3dcbb851fd4a2f31368e1229482ab076090"}
Jul 30 19:24:43.790060 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.790038    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p7ltn" event={"ID":"15de2c06-37ea-4122-9eaf-5fbe481c3127","Type":"ContainerStarted","Data":"ed8b41316f95587f20fd7f20c86fbffa58c0d16d4c378a5f1a5a4b1a48175c4c"}
Jul 30 19:24:43.791670 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.791650    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" event={"ID":"c7684a39bc1278f7d06d66597af84743","Type":"ContainerStarted","Data":"5b054a26caed8b43f88039106647c89d1e26b517363418e5786deade6541789b"}
Jul 30 19:24:43.792679 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.792656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4b287" event={"ID":"276fa5f2-21c9-42fc-88fc-6f7d3db559c9","Type":"ContainerStarted","Data":"66e98c75cf6075df7d9920698675a5838fed73e0c8c06c9ec8892c9fb79acb84"}
Jul 30 19:24:43.793733 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.793712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc" event={"ID":"01798510-b255-4582-adb9-866c4fbc4bb6","Type":"ContainerStarted","Data":"890f9b70b17dd0eb22bc71d8f9e0292cd539e769bf70c6dbe54351153d341b80"}
Jul 30 19:24:43.794828 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.794801    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7j96x" event={"ID":"3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1","Type":"ContainerStarted","Data":"cc4ca4d8806f9b2ff81d30c3c62546096edecb3d76835376e44d13c7c32a8d9e"}
Jul 30 19:24:43.795867 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.795847    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9b6xj" event={"ID":"7b945f77-954b-47f9-a8db-8c6c586dc346","Type":"ContainerStarted","Data":"954e086df7362c83ba9ab38310de54b3f7c5b1bc8ffe5442ee0148ca364e7dd1"}
Jul 30 19:24:43.796821 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.796800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bpbdk" event={"ID":"88cc7cdc-3575-4a0b-b5f3-740fb512673b","Type":"ContainerStarted","Data":"4a6e46687cc643ec9e4645e720ec515b8f2bd3f9d2f2187709f9e5610efbd0c4"}
Jul 30 19:24:43.804155 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:43.804100    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" podStartSLOduration=2.804089588 podStartE2EDuration="2.804089588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:24:43.803971878 +0000 UTC m=+3.564727137" watchObservedRunningTime="2026-07-30 19:24:43.804089588 +0000 UTC m=+3.564844837"
Jul 30 19:24:44.233658 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:44.233577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:44.233804 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:44.233737    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:44.233804 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:44.233802    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret podName:3fe769f8-bc2c-466c-be78-95ef7f74263c nodeName:}" failed. No retries permitted until 2026-07-30 19:24:46.233782244 +0000 UTC m=+5.994537503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret") pod "global-pull-secret-syncer-7dmqn" (UID: "3fe769f8-bc2c-466c-be78-95ef7f74263c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:44.334945 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:44.334397    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:44.334945 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:44.334569    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:44.334945 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:44.334629    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs podName:99fc519b-8e3c-4564-a65a-f81a314974f9 nodeName:}" failed. No retries permitted until 2026-07-30 19:24:46.334610293 +0000 UTC m=+6.095365534 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs") pod "network-metrics-daemon-26gjf" (UID: "99fc519b-8e3c-4564-a65a-f81a314974f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:44.434895 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:44.434859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsx9g\" (UniqueName: \"kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g\") pod \"network-check-target-8ml9s\" (UID: \"6e6a374a-1f04-4351-bb56-8e329d65bc25\") " pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:44.435062 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:44.435051    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:44.435195 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:44.435088    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:44.435195 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:44.435102    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lsx9g for pod openshift-network-diagnostics/network-check-target-8ml9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:44.435195 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:44.435160    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g podName:6e6a374a-1f04-4351-bb56-8e329d65bc25 nodeName:}" failed. No retries permitted until 2026-07-30 19:24:46.435142861 +0000 UTC m=+6.195898120 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsx9g" (UniqueName: "kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g") pod "network-check-target-8ml9s" (UID: "6e6a374a-1f04-4351-bb56-8e329d65bc25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:44.645776 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:44.645639    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 19:19:42 +0000 UTC" deadline="2028-05-16 11:56:40.266539974 +0000 UTC"
Jul 30 19:24:44.645776 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:44.645701    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15736h31m55.620842903s"
Jul 30 19:24:44.781576 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:44.780897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:44.781576 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:44.781013    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:24:44.781576 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:44.781406    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:44.781576 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:44.781496    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:24:44.815485 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:44.813434    2577 generic.go:356] "Generic (PLEG): container finished" podID="df1f3ece247108934ba885a4bf3aae23" containerID="8eaf4fb2097f622b56cc881dfa0162660dc1eaf0a8fcd6b0c51dc45d5ed0e627" exitCode=0
Jul 30 19:24:44.815485 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:44.814135    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" event={"ID":"df1f3ece247108934ba885a4bf3aae23","Type":"ContainerDied","Data":"8eaf4fb2097f622b56cc881dfa0162660dc1eaf0a8fcd6b0c51dc45d5ed0e627"}
Jul 30 19:24:45.779022 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:45.778989    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:45.779511 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:45.779144    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:24:45.823886 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:45.823827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" event={"ID":"df1f3ece247108934ba885a4bf3aae23","Type":"ContainerStarted","Data":"29b0c482bcb398eebc0965a9fc184e296e85f8c0d18335c5f2f5c1050af7f665"}
Jul 30 19:24:45.841323 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:45.840784    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" podStartSLOduration=4.840765139 podStartE2EDuration="4.840765139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:24:45.840319517 +0000 UTC m=+5.601074776" watchObservedRunningTime="2026-07-30 19:24:45.840765139 +0000 UTC m=+5.601520397"
Jul 30 19:24:46.250765 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:46.250262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:46.250765 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:46.250392    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:46.250765 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:46.250445    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret podName:3fe769f8-bc2c-466c-be78-95ef7f74263c nodeName:}" failed. No retries permitted until 2026-07-30 19:24:50.250432341 +0000 UTC m=+10.011187576 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret") pod "global-pull-secret-syncer-7dmqn" (UID: "3fe769f8-bc2c-466c-be78-95ef7f74263c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:46.351481 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:46.350771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:46.351481 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:46.350956    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:46.351481 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:46.351016    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs podName:99fc519b-8e3c-4564-a65a-f81a314974f9 nodeName:}" failed. No retries permitted until 2026-07-30 19:24:50.35099909 +0000 UTC m=+10.111754346 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs") pod "network-metrics-daemon-26gjf" (UID: "99fc519b-8e3c-4564-a65a-f81a314974f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:46.451735 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:46.451699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsx9g\" (UniqueName: \"kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g\") pod \"network-check-target-8ml9s\" (UID: \"6e6a374a-1f04-4351-bb56-8e329d65bc25\") " pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:46.451906 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:46.451862    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:46.451906 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:46.451922    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:46.452028 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:46.451943    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lsx9g for pod openshift-network-diagnostics/network-check-target-8ml9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:46.452028 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:46.452002    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g podName:6e6a374a-1f04-4351-bb56-8e329d65bc25 nodeName:}" failed. No retries permitted until 2026-07-30 19:24:50.451983333 +0000 UTC m=+10.212738644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsx9g" (UniqueName: "kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g") pod "network-check-target-8ml9s" (UID: "6e6a374a-1f04-4351-bb56-8e329d65bc25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:46.779320 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:46.779282    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:46.779730 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:46.779428    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:24:46.779730 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:46.779532    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:46.779730 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:46.779645    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:24:47.779318 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:47.779261    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:47.779503 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:47.779439    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:24:48.778565 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:48.778509    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:48.778565 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:48.778559    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:48.778787 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:48.778636    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:24:48.778787 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:48.778721    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:24:49.778506 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:49.778470    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:49.778960 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:49.778605    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:24:50.283565 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:50.283533    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:50.283764 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:50.283681    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:50.283764 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:50.283756    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret podName:3fe769f8-bc2c-466c-be78-95ef7f74263c nodeName:}" failed. No retries permitted until 2026-07-30 19:24:58.283731672 +0000 UTC m=+18.044486914 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret") pod "global-pull-secret-syncer-7dmqn" (UID: "3fe769f8-bc2c-466c-be78-95ef7f74263c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:50.384376 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:50.384305    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:50.384540 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:50.384430    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:50.384540 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:50.384498    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs podName:99fc519b-8e3c-4564-a65a-f81a314974f9 nodeName:}" failed. No retries permitted until 2026-07-30 19:24:58.38447824 +0000 UTC m=+18.145233495 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs") pod "network-metrics-daemon-26gjf" (UID: "99fc519b-8e3c-4564-a65a-f81a314974f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:50.484807 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:50.484772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsx9g\" (UniqueName: \"kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g\") pod \"network-check-target-8ml9s\" (UID: \"6e6a374a-1f04-4351-bb56-8e329d65bc25\") " pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:50.484966 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:50.484900    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:50.484966 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:50.484916    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:50.484966 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:50.484924    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lsx9g for pod openshift-network-diagnostics/network-check-target-8ml9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:50.484966 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:50.484963    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g podName:6e6a374a-1f04-4351-bb56-8e329d65bc25 nodeName:}" failed. No retries permitted until 2026-07-30 19:24:58.484951125 +0000 UTC m=+18.245706362 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsx9g" (UniqueName: "kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g") pod "network-check-target-8ml9s" (UID: "6e6a374a-1f04-4351-bb56-8e329d65bc25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:50.780291 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:50.780213    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:50.780668 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:50.780318    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:24:50.780762 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:50.780743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:50.780878 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:50.780844    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:24:51.779152 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:51.778645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:51.779152 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:51.778786    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:24:52.779348 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:52.778917    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:52.779348 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:52.779022    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:24:52.779348 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:52.779098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:52.779348 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:52.779222    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:24:53.779162 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:53.779136    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:53.779331 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:53.779310    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:24:54.779181 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:54.779153    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:54.779557 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:54.779257    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:24:54.779557 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:54.779287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:54.779557 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:54.779354    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:24:55.778733 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:55.778701    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:55.778914 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:55.778825    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:24:56.778583 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:56.778549    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:56.779028 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:56.778686    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:24:56.779028 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:56.778751    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:56.779028 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:56.778858    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:24:57.779043 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:57.779017    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:57.779392 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:57.779126    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:24:58.345756 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:58.345717    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:58.345937 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:58.345867    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:58.345988 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:58.345942    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret podName:3fe769f8-bc2c-466c-be78-95ef7f74263c nodeName:}" failed. No retries permitted until 2026-07-30 19:25:14.345922384 +0000 UTC m=+34.106677620 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret") pod "global-pull-secret-syncer-7dmqn" (UID: "3fe769f8-bc2c-466c-be78-95ef7f74263c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:58.446826 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:58.446800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:58.446958 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:58.446891    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:58.446958 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:58.446948    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs podName:99fc519b-8e3c-4564-a65a-f81a314974f9 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:14.446929414 +0000 UTC m=+34.207684655 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs") pod "network-metrics-daemon-26gjf" (UID: "99fc519b-8e3c-4564-a65a-f81a314974f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:58.547127 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:58.547094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsx9g\" (UniqueName: \"kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g\") pod \"network-check-target-8ml9s\" (UID: \"6e6a374a-1f04-4351-bb56-8e329d65bc25\") " pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:58.547290 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:58.547271    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:58.547356 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:58.547295    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:58.547356 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:58.547307    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lsx9g for pod openshift-network-diagnostics/network-check-target-8ml9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:58.547460 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:58.547371    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g podName:6e6a374a-1f04-4351-bb56-8e329d65bc25 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:14.547352718 +0000 UTC m=+34.308107956 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsx9g" (UniqueName: "kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g") pod "network-check-target-8ml9s" (UID: "6e6a374a-1f04-4351-bb56-8e329d65bc25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:58.778728 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:58.778656    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:24:58.778889 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:58.778761    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:24:58.778889 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:58.778847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:24:58.778990 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:58.778948    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:24:59.779018 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:24:59.778983    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:24:59.779409 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:24:59.779148    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:25:00.779931 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:00.779886    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:00.780396 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:00.780004    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:25:00.780396 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:00.780063    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:00.780396 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:00.780213    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:25:01.778655 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:01.778627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:01.778836 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:01.778720    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:25:02.779061 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.778653    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:02.779773 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.778662    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:02.779773 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:02.779175    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:25:02.779773 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:02.779258    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:25:02.858651 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.858620    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4b287" event={"ID":"276fa5f2-21c9-42fc-88fc-6f7d3db559c9","Type":"ContainerStarted","Data":"42ea4b32039ca64841dc3c1efff060664cdd35fb2d1bdbbf7decba69afbefd1f"}
Jul 30 19:25:02.862132 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.862107    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc" event={"ID":"01798510-b255-4582-adb9-866c4fbc4bb6","Type":"ContainerStarted","Data":"53469b2ba8f1bbefab3711e0d21bc0950261445d16dad2a30d081d29eb73a06b"}
Jul 30 19:25:02.862269 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.862180    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc" event={"ID":"01798510-b255-4582-adb9-866c4fbc4bb6","Type":"ContainerStarted","Data":"4e62dcaca74e19607cd5123ce331531db45fb8943a6016b53916b5bc37d7544d"}
Jul 30 19:25:02.862269 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.862230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc" event={"ID":"01798510-b255-4582-adb9-866c4fbc4bb6","Type":"ContainerStarted","Data":"1753496d2db9db7087f1ed908bb6f3fd9623c065732ed96f0227724831f2452e"}
Jul 30 19:25:02.862396 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.862306    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc" event={"ID":"01798510-b255-4582-adb9-866c4fbc4bb6","Type":"ContainerStarted","Data":"43d9472a26eb7001c1a775621a4e7acb3d4646ef893b42fb6b9704655be8825d"}
Jul 30 19:25:02.862396 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.862358    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc" event={"ID":"01798510-b255-4582-adb9-866c4fbc4bb6","Type":"ContainerStarted","Data":"83490a06f6eaf390d2936d5ec40676f248d95f1c0c3f96bb67440298ad0e3592"}
Jul 30 19:25:02.862489 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.862406    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc" event={"ID":"01798510-b255-4582-adb9-866c4fbc4bb6","Type":"ContainerStarted","Data":"5eca369ad03a5e2febadb0f6783fa13e356e4d2f73b17a607cf48eb7fe26ce8e"}
Jul 30 19:25:02.863888 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.863837    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7j96x" event={"ID":"3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1","Type":"ContainerStarted","Data":"ce2c2407dba309a17a2984cadc3a7eb8ac37fa206a19f4e0021d9306e1be0222"}
Jul 30 19:25:02.865664 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.865637    2577 generic.go:356] "Generic (PLEG): container finished" podID="7b945f77-954b-47f9-a8db-8c6c586dc346" containerID="c27fd0e16c9cb8a58021fe9253d51c15e72643120123b27df368c6b13a0cefd2" exitCode=0
Jul 30 19:25:02.865754 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.865712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9b6xj" event={"ID":"7b945f77-954b-47f9-a8db-8c6c586dc346","Type":"ContainerDied","Data":"c27fd0e16c9cb8a58021fe9253d51c15e72643120123b27df368c6b13a0cefd2"}
Jul 30 19:25:02.867517 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.867492    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pcl9g" event={"ID":"f40f7c70-244b-4c1f-9d41-9485539a4ba2","Type":"ContainerStarted","Data":"3d273d822c97d6c9d6a238404a66103f6d4b53a2ac26d2e60f521581e0ef5794"}
Jul 30 19:25:02.869340 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.869316    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tgv2k" event={"ID":"0880f05e-770d-46dd-a957-08c121ec9fac","Type":"ContainerStarted","Data":"dda858793f7658127150ac5b9c0494d76c79b3bd1241e9ad130f08473cbfd7ca"}
Jul 30 19:25:02.871997 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.871975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg" event={"ID":"25ca9a86-bd6a-4728-adb6-954f15ebb254","Type":"ContainerStarted","Data":"c0b247b45aacac8beac055042b223ced7c5d3d8a177d1581134d9223e9d5c7a1"}
Jul 30 19:25:02.873872 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.873848    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p7ltn" event={"ID":"15de2c06-37ea-4122-9eaf-5fbe481c3127","Type":"ContainerStarted","Data":"0484163a360d9f0431d275c256c9e100ac649c221a8b96fd6d5e3de017e9fccc"}
Jul 30 19:25:02.876374 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.876330    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4b287" podStartSLOduration=4.495548077 podStartE2EDuration="22.876313367s" totalImagesPullingTime="18.38076529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:02.875711162 +0000 UTC m=+22.636466411" watchObservedRunningTime="2026-07-30 19:25:02.876313367 +0000 UTC m=+22.637068628"
Jul 30 19:25:02.909375 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.909339    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7j96x" podStartSLOduration=4.525466192 podStartE2EDuration="22.909328119s" totalImagesPullingTime="18.383861927s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:02.908881917 +0000 UTC m=+22.669637176" watchObservedRunningTime="2026-07-30 19:25:02.909328119 +0000 UTC m=+22.670083398"
Jul 30 19:25:02.924198 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.924167    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pcl9g" podStartSLOduration=4.513034045 podStartE2EDuration="22.924156062s" totalImagesPullingTime="18.411122017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:02.924009906 +0000 UTC m=+22.684765186" watchObservedRunningTime="2026-07-30 19:25:02.924156062 +0000 UTC m=+22.684911320"
Jul 30 19:25:02.943390 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.943358    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tgv2k" podStartSLOduration=4.493534847 podStartE2EDuration="22.943344184s" totalImagesPullingTime="18.449809337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:02.943023448 +0000 UTC m=+22.703778696" watchObservedRunningTime="2026-07-30 19:25:02.943344184 +0000 UTC m=+22.704099444"
Jul 30 19:25:02.959647 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:02.959615    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p7ltn" podStartSLOduration=8.829385656 podStartE2EDuration="22.959605043s" totalImagesPullingTime="14.130219387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:02.959270305 +0000 UTC m=+22.720025554" watchObservedRunningTime="2026-07-30 19:25:02.959605043 +0000 UTC m=+22.720360301"
Jul 30 19:25:03.530835 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:03.530812    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 19:25:03.679183 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:03.679017    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T19:25:03.530829951Z","UUID":"a7ec274b-99f0-41cc-9455-f374dfed756b","Handler":null,"Name":"","Endpoint":""}
Jul 30 19:25:03.682054 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:03.682008    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 19:25:03.682054 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:03.682054    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 19:25:03.779258 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:03.779200    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:03.779778 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:03.779344    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:25:03.877915 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:03.877883    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bpbdk" event={"ID":"88cc7cdc-3575-4a0b-b5f3-740fb512673b","Type":"ContainerStarted","Data":"58f9cb161d3b7716be0ef34e27e8e912abcbb7d4e92d2d706855e2e87b11e005"}
Jul 30 19:25:03.879956 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:03.879927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg" event={"ID":"25ca9a86-bd6a-4728-adb6-954f15ebb254","Type":"ContainerStarted","Data":"a8f52e8a13a4dc1a83cf2020cfff1f051efd6058546c4ba13dab0e3fa96e1e2c"}
Jul 30 19:25:03.892757 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:03.892709    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bpbdk" podStartSLOduration=5.48250279 podStartE2EDuration="23.892691689s" totalImagesPullingTime="18.410188899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:03.892016371 +0000 UTC m=+23.652771643" watchObservedRunningTime="2026-07-30 19:25:03.892691689 +0000 UTC m=+23.653446948"
Jul 30 19:25:04.778308 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:04.778281    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:04.778308 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:04.778300    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:04.778501 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:04.778400    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:25:04.778565 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:04.778546    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:25:05.383418 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:05.383258    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:25:05.384126 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:05.384105    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:25:05.778939 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:05.778866    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:05.779092 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:05.779002    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:25:05.890968 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:05.890926    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc" event={"ID":"01798510-b255-4582-adb9-866c4fbc4bb6","Type":"ContainerStarted","Data":"a4fec359c5fba282390a6d6fcdb36ae9a1c329847bce9ae104798422568393f2"}
Jul 30 19:25:05.893422 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:05.893391    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg" event={"ID":"25ca9a86-bd6a-4728-adb6-954f15ebb254","Type":"ContainerStarted","Data":"34d16079b5f99ee95b6d9aba3479e7457326cabed4eba797d00ed67941037420"}
Jul 30 19:25:05.911931 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:05.911889    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4nvg" podStartSLOduration=5.030673239 podStartE2EDuration="25.911877944s" totalImagesPullingTime="20.881204705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:05.91135926 +0000 UTC m=+25.672114517" watchObservedRunningTime="2026-07-30 19:25:05.911877944 +0000 UTC m=+25.672633202"
Jul 30 19:25:06.778993 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:06.778962    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:06.779518 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:06.779102    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:25:06.779518 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:06.779142    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:06.779518 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:06.779244    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:25:06.982347 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:06.982320    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:25:06.982502 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:06.982413    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 30 19:25:06.982978 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:06.982959    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p7ltn"
Jul 30 19:25:07.778636 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:07.778483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:07.778769 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:07.778709    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:25:07.900768 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:07.900739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc" event={"ID":"01798510-b255-4582-adb9-866c4fbc4bb6","Type":"ContainerStarted","Data":"a156ceed49a7e21d1054c462433a3d48b6e299f123b1ac0945ade066a70771a8"}
Jul 30 19:25:07.901388 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:07.900920    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:25:07.902612 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:07.902588    2577 generic.go:356] "Generic (PLEG): container finished" podID="7b945f77-954b-47f9-a8db-8c6c586dc346" containerID="8c6b2c9e22985a78dd6dac3bb5a1cff6a42e02fafef18dd8b6970c3c1b049367" exitCode=0
Jul 30 19:25:07.902721 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:07.902666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9b6xj" event={"ID":"7b945f77-954b-47f9-a8db-8c6c586dc346","Type":"ContainerDied","Data":"8c6b2c9e22985a78dd6dac3bb5a1cff6a42e02fafef18dd8b6970c3c1b049367"}
Jul 30 19:25:07.916665 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:07.916648    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:25:07.925007 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:07.924974    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc" podStartSLOduration=9.342267919 podStartE2EDuration="27.924961728s" totalImagesPullingTime="18.582693809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:07.923885191 +0000 UTC m=+27.684640448" watchObservedRunningTime="2026-07-30 19:25:07.924961728 +0000 UTC m=+27.685716983"
Jul 30 19:25:08.778547 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:08.778517    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:08.778687 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:08.778556    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:08.778687 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:08.778638    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:25:08.778768 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:08.778748    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:25:08.906171 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:08.906002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:25:08.906171 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:08.906129    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:25:08.922444 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:08.922415    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:25:09.178345 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:09.178278    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7dmqn"]
Jul 30 19:25:09.178468 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:09.178383    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:09.178529 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:09.178487    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:25:09.185981 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:09.185957    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8ml9s"]
Jul 30 19:25:09.186100 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:09.186048    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:09.186190 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:09.186154    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:25:09.186764 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:09.186741    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-26gjf"]
Jul 30 19:25:09.186862 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:09.186850    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:09.186942 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:09.186926    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:25:09.909049 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:09.909012    2577 generic.go:356] "Generic (PLEG): container finished" podID="7b945f77-954b-47f9-a8db-8c6c586dc346" containerID="4c5d4bf4e38856ca1ef368d88b3e6e9377f901774bb149f6c0f8be9b210e23a6" exitCode=0
Jul 30 19:25:09.909483 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:09.909064    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9b6xj" event={"ID":"7b945f77-954b-47f9-a8db-8c6c586dc346","Type":"ContainerDied","Data":"4c5d4bf4e38856ca1ef368d88b3e6e9377f901774bb149f6c0f8be9b210e23a6"}
Jul 30 19:25:10.779170 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:10.779141    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:10.779317 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:10.779216    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:25:10.779317 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:10.779259    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:10.779317 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:10.779307    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:25:10.779452 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:10.779336    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:10.779452 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:10.779378    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:25:11.916158 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:11.916120    2577 generic.go:356] "Generic (PLEG): container finished" podID="7b945f77-954b-47f9-a8db-8c6c586dc346" containerID="31cf32a53cc135993c16a33cc9897e59f30a0688ccc7b7f896c91c994ba978fa" exitCode=0
Jul 30 19:25:11.916742 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:11.916174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9b6xj" event={"ID":"7b945f77-954b-47f9-a8db-8c6c586dc346","Type":"ContainerDied","Data":"31cf32a53cc135993c16a33cc9897e59f30a0688ccc7b7f896c91c994ba978fa"}
Jul 30 19:25:12.778456 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:12.778426    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:12.778609 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:12.778459    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:12.778609 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:12.778433    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:12.778609 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:12.778544    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ml9s" podUID="6e6a374a-1f04-4351-bb56-8e329d65bc25"
Jul 30 19:25:12.778773 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:12.778639    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7dmqn" podUID="3fe769f8-bc2c-466c-be78-95ef7f74263c"
Jul 30 19:25:12.778773 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:12.778739    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26gjf" podUID="99fc519b-8e3c-4564-a65a-f81a314974f9"
Jul 30 19:25:14.366277 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.366240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:14.366935 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:14.366396    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:25:14.366935 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:14.366469    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret podName:3fe769f8-bc2c-466c-be78-95ef7f74263c nodeName:}" failed. No retries permitted until 2026-07-30 19:25:46.366450617 +0000 UTC m=+66.127205853 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret") pod "global-pull-secret-syncer-7dmqn" (UID: "3fe769f8-bc2c-466c-be78-95ef7f74263c") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:25:14.466856 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.466826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:14.466997 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:14.466945    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:25:14.467049 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:14.467000    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs podName:99fc519b-8e3c-4564-a65a-f81a314974f9 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:46.466985034 +0000 UTC m=+66.227740270 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs") pod "network-metrics-daemon-26gjf" (UID: "99fc519b-8e3c-4564-a65a-f81a314974f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:25:14.567869 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.567834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsx9g\" (UniqueName: \"kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g\") pod \"network-check-target-8ml9s\" (UID: \"6e6a374a-1f04-4351-bb56-8e329d65bc25\") " pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:14.568016 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:14.567998    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:25:14.568085 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:14.568021    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:25:14.568085 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:14.568031    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lsx9g for pod openshift-network-diagnostics/network-check-target-8ml9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:25:14.568162 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:14.568099    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g podName:6e6a374a-1f04-4351-bb56-8e329d65bc25 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:46.568080606 +0000 UTC m=+66.328835856 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lsx9g" (UniqueName: "kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g") pod "network-check-target-8ml9s" (UID: "6e6a374a-1f04-4351-bb56-8e329d65bc25") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:25:14.610689 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.610666    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeReady"
Jul 30 19:25:14.610813 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.610790    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 19:25:14.652227 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.652164    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7mqgt"]
Jul 30 19:25:14.655164 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.654996    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.655496 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.655414    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ftznv"]
Jul 30 19:25:14.657185 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.657162    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9htd7\""
Jul 30 19:25:14.657594 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.657296    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ftznv"
Jul 30 19:25:14.657594 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.657369    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 19:25:14.657594 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.657518    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 19:25:14.659735 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.659714    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 19:25:14.660241 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.660013    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 19:25:14.660241 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.660052    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 19:25:14.660410 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.660393    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8dddr\""
Jul 30 19:25:14.665081 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.665044    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7mqgt"]
Jul 30 19:25:14.666563 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.666516    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ftznv"]
Jul 30 19:25:14.769405 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.769369    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c153fb6e-27af-4e8e-a363-1bc0922c458e-tmp-dir\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.769405 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.769408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q644f\" (UniqueName: \"kubernetes.io/projected/c153fb6e-27af-4e8e-a363-1bc0922c458e-kube-api-access-q644f\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.769589 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.769426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a958f2b4-3331-4143-8794-7297a6e533ec-cert\") pod \"ingress-canary-ftznv\" (UID: \"a958f2b4-3331-4143-8794-7297a6e533ec\") " pod="openshift-ingress-canary/ingress-canary-ftznv"
Jul 30 19:25:14.769589 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.769496    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccrq6\" (UniqueName: \"kubernetes.io/projected/a958f2b4-3331-4143-8794-7297a6e533ec-kube-api-access-ccrq6\") pod \"ingress-canary-ftznv\" (UID: \"a958f2b4-3331-4143-8794-7297a6e533ec\") " pod="openshift-ingress-canary/ingress-canary-ftznv"
Jul 30 19:25:14.769589 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.769568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c153fb6e-27af-4e8e-a363-1bc0922c458e-config-volume\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.769717 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.769633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.781794 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.781768    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:14.781913 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.781798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:14.781913 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.781874    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:14.784488 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.784317    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bgfkv\""
Jul 30 19:25:14.784488 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.784336    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-82x98\""
Jul 30 19:25:14.784488 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.784361    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 19:25:14.784488 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.784419    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 19:25:14.784488 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.784436    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 19:25:14.784488 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.784324    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 19:25:14.870661 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.870631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccrq6\" (UniqueName: \"kubernetes.io/projected/a958f2b4-3331-4143-8794-7297a6e533ec-kube-api-access-ccrq6\") pod \"ingress-canary-ftznv\" (UID: \"a958f2b4-3331-4143-8794-7297a6e533ec\") " pod="openshift-ingress-canary/ingress-canary-ftznv"
Jul 30 19:25:14.870782 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.870673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c153fb6e-27af-4e8e-a363-1bc0922c458e-config-volume\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.870782 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.870701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.870782 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.870761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c153fb6e-27af-4e8e-a363-1bc0922c458e-tmp-dir\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.870935 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.870786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q644f\" (UniqueName: \"kubernetes.io/projected/c153fb6e-27af-4e8e-a363-1bc0922c458e-kube-api-access-q644f\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.870935 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.870807    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a958f2b4-3331-4143-8794-7297a6e533ec-cert\") pod \"ingress-canary-ftznv\" (UID: \"a958f2b4-3331-4143-8794-7297a6e533ec\") " pod="openshift-ingress-canary/ingress-canary-ftznv"
Jul 30 19:25:14.871014 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:14.870929    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:14.871054 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:14.871020    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls podName:c153fb6e-27af-4e8e-a363-1bc0922c458e nodeName:}" failed. No retries permitted until 2026-07-30 19:25:15.370994551 +0000 UTC m=+35.131749811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls") pod "dns-default-7mqgt" (UID: "c153fb6e-27af-4e8e-a363-1bc0922c458e") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:14.871287 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.871244    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c153fb6e-27af-4e8e-a363-1bc0922c458e-tmp-dir\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.871287 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.871273    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c153fb6e-27af-4e8e-a363-1bc0922c458e-config-volume\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.876228 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.876206    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a958f2b4-3331-4143-8794-7297a6e533ec-cert\") pod \"ingress-canary-ftznv\" (UID: \"a958f2b4-3331-4143-8794-7297a6e533ec\") " pod="openshift-ingress-canary/ingress-canary-ftznv"
Jul 30 19:25:14.881295 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.881264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccrq6\" (UniqueName: \"kubernetes.io/projected/a958f2b4-3331-4143-8794-7297a6e533ec-kube-api-access-ccrq6\") pod \"ingress-canary-ftznv\" (UID: \"a958f2b4-3331-4143-8794-7297a6e533ec\") " pod="openshift-ingress-canary/ingress-canary-ftznv"
Jul 30 19:25:14.881733 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.881711    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q644f\" (UniqueName: \"kubernetes.io/projected/c153fb6e-27af-4e8e-a363-1bc0922c458e-kube-api-access-q644f\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:14.974021 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:14.973962    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ftznv"
Jul 30 19:25:15.375390 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:15.375170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:15.375845 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:15.375325    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:15.375845 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:15.375501    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls podName:c153fb6e-27af-4e8e-a363-1bc0922c458e nodeName:}" failed. No retries permitted until 2026-07-30 19:25:16.375485911 +0000 UTC m=+36.136241166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls") pod "dns-default-7mqgt" (UID: "c153fb6e-27af-4e8e-a363-1bc0922c458e") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:15.817371 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:15.817348    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ftznv"]
Jul 30 19:25:15.821103 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:25:15.821081    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda958f2b4_3331_4143_8794_7297a6e533ec.slice/crio-bcaf2dfe61ce2119f6649e6d25c88b8bccd1f7770bf285064f47906b719509f7 WatchSource:0}: Error finding container bcaf2dfe61ce2119f6649e6d25c88b8bccd1f7770bf285064f47906b719509f7: Status 404 returned error can't find the container with id bcaf2dfe61ce2119f6649e6d25c88b8bccd1f7770bf285064f47906b719509f7
Jul 30 19:25:15.927474 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:15.927446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ftznv" event={"ID":"a958f2b4-3331-4143-8794-7297a6e533ec","Type":"ContainerStarted","Data":"bcaf2dfe61ce2119f6649e6d25c88b8bccd1f7770bf285064f47906b719509f7"}
Jul 30 19:25:15.930117 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:15.930096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9b6xj" event={"ID":"7b945f77-954b-47f9-a8db-8c6c586dc346","Type":"ContainerStarted","Data":"5423a52cb2878356dfa63bb2f78a4424c81a181d19ca9be9925e6f97640dd7d4"}
Jul 30 19:25:16.382669 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:16.382632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:16.383055 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:16.382773    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:16.383055 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:16.382836    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls podName:c153fb6e-27af-4e8e-a363-1bc0922c458e nodeName:}" failed. No retries permitted until 2026-07-30 19:25:18.382818355 +0000 UTC m=+38.143573591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls") pod "dns-default-7mqgt" (UID: "c153fb6e-27af-4e8e-a363-1bc0922c458e") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:16.935990 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:16.935955    2577 generic.go:356] "Generic (PLEG): container finished" podID="7b945f77-954b-47f9-a8db-8c6c586dc346" containerID="5423a52cb2878356dfa63bb2f78a4424c81a181d19ca9be9925e6f97640dd7d4" exitCode=0
Jul 30 19:25:16.935990 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:16.935985    2577 generic.go:356] "Generic (PLEG): container finished" podID="7b945f77-954b-47f9-a8db-8c6c586dc346" containerID="429e6541dff1f5d7f9f2020a6ebfa7c64e6517dc169f59ac390521c1fb25e738" exitCode=0
Jul 30 19:25:16.936242 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:16.936039    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9b6xj" event={"ID":"7b945f77-954b-47f9-a8db-8c6c586dc346","Type":"ContainerDied","Data":"5423a52cb2878356dfa63bb2f78a4424c81a181d19ca9be9925e6f97640dd7d4"}
Jul 30 19:25:16.936242 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:16.936122    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9b6xj" event={"ID":"7b945f77-954b-47f9-a8db-8c6c586dc346","Type":"ContainerDied","Data":"429e6541dff1f5d7f9f2020a6ebfa7c64e6517dc169f59ac390521c1fb25e738"}
Jul 30 19:25:17.941266 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:17.941099    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9b6xj" event={"ID":"7b945f77-954b-47f9-a8db-8c6c586dc346","Type":"ContainerStarted","Data":"7df327dd3c4258db2a039154115ed0c558bd03235eb19717efd9b107a3588a68"}
Jul 30 19:25:17.967476 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:17.967429    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9b6xj" podStartSLOduration=9.426013007 podStartE2EDuration="37.967414816s" totalImagesPullingTime="28.541401809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:17.966938025 +0000 UTC m=+37.727693299" watchObservedRunningTime="2026-07-30 19:25:17.967414816 +0000 UTC m=+37.728170073"
Jul 30 19:25:18.397109 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:18.397061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:18.397249 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:18.397193    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:18.397299 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:18.397273    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls podName:c153fb6e-27af-4e8e-a363-1bc0922c458e nodeName:}" failed. No retries permitted until 2026-07-30 19:25:22.397251808 +0000 UTC m=+42.158007068 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls") pod "dns-default-7mqgt" (UID: "c153fb6e-27af-4e8e-a363-1bc0922c458e") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:18.947217 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:18.947176    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ftznv" event={"ID":"a958f2b4-3331-4143-8794-7297a6e533ec","Type":"ContainerStarted","Data":"19c6e1230020e7fa7ae67dafd5151ae0b3c463d98dbb7f8c443fd021264d1f82"}
Jul 30 19:25:18.961150 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:18.961108    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ftznv" podStartSLOduration=2.799675296 podStartE2EDuration="4.961095885s" totalImagesPullingTime="2.161420589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:18.961086098 +0000 UTC m=+38.721841358" watchObservedRunningTime="2026-07-30 19:25:18.961095885 +0000 UTC m=+38.721851143"
Jul 30 19:25:19.897228 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:19.897198    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc"]
Jul 30 19:25:19.921640 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:19.921614    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc"]
Jul 30 19:25:19.921756 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:19.921701    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc"
Jul 30 19:25:19.923857 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:19.923840    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 30 19:25:19.923947 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:19.923877    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 30 19:25:19.924676 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:19.924659    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5r87b\""
Jul 30 19:25:20.107986 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.107949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t75ch\" (UniqueName: \"kubernetes.io/projected/1eee9ece-e46a-432a-aced-5004c9ed090b-kube-api-access-t75ch\") pod \"migrator-68bc745d87-76dtc\" (UID: \"1eee9ece-e46a-432a-aced-5004c9ed090b\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc"
Jul 30 19:25:20.151628 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.151547    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s47sv"]
Jul 30 19:25:20.170717 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.170689    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s47sv"]
Jul 30 19:25:20.170823 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.170804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.173044 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.173023    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 19:25:20.173173 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.173133    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 19:25:20.173173 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.173149    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7gsjq\""
Jul 30 19:25:20.173896 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.173881    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 19:25:20.173981 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.173883    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 19:25:20.208546 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.208523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t75ch\" (UniqueName: \"kubernetes.io/projected/1eee9ece-e46a-432a-aced-5004c9ed090b-kube-api-access-t75ch\") pod \"migrator-68bc745d87-76dtc\" (UID: \"1eee9ece-e46a-432a-aced-5004c9ed090b\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc"
Jul 30 19:25:20.218975 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.218956    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t75ch\" (UniqueName: \"kubernetes.io/projected/1eee9ece-e46a-432a-aced-5004c9ed090b-kube-api-access-t75ch\") pod \"migrator-68bc745d87-76dtc\" (UID: \"1eee9ece-e46a-432a-aced-5004c9ed090b\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc"
Jul 30 19:25:20.230766 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.230750    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc"
Jul 30 19:25:20.309706 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.309674    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/288e7048-1964-4683-ab92-3fcacea15656-data-volume\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.309825 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.309727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.309897 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.309838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhp9s\" (UniqueName: \"kubernetes.io/projected/288e7048-1964-4683-ab92-3fcacea15656-kube-api-access-fhp9s\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.309948 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.309919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/288e7048-1964-4683-ab92-3fcacea15656-crio-socket\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.310035 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.310004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.367807 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.367782    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc"]
Jul 30 19:25:20.370601 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:25:20.370576    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eee9ece_e46a_432a_aced_5004c9ed090b.slice/crio-769d3db175e3c779f82c2c990fe06b2113a10656a23470f3384e59d9600f9940 WatchSource:0}: Error finding container 769d3db175e3c779f82c2c990fe06b2113a10656a23470f3384e59d9600f9940: Status 404 returned error can't find the container with id 769d3db175e3c779f82c2c990fe06b2113a10656a23470f3384e59d9600f9940
Jul 30 19:25:20.410858 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.410810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/288e7048-1964-4683-ab92-3fcacea15656-data-volume\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.410858 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.410836    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.410963 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.410864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhp9s\" (UniqueName: \"kubernetes.io/projected/288e7048-1964-4683-ab92-3fcacea15656-kube-api-access-fhp9s\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.411001 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.410962    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/288e7048-1964-4683-ab92-3fcacea15656-crio-socket\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.411001 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:20.410971    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:20.411001 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.410992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.411132 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:20.411044    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls podName:288e7048-1964-4683-ab92-3fcacea15656 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:20.911024295 +0000 UTC m=+40.671779550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s47sv" (UID: "288e7048-1964-4683-ab92-3fcacea15656") : secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:20.411132 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:20.411065    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 19:25:20.411213 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:20.411136    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm podName:288e7048-1964-4683-ab92-3fcacea15656 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:20.911121255 +0000 UTC m=+40.671876511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s47sv" (UID: "288e7048-1964-4683-ab92-3fcacea15656") : configmap "kube-rbac-proxy" not found
Jul 30 19:25:20.411213 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.411189    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/288e7048-1964-4683-ab92-3fcacea15656-data-volume\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.411283 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.411245    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/288e7048-1964-4683-ab92-3fcacea15656-crio-socket\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.418724 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.418700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhp9s\" (UniqueName: \"kubernetes.io/projected/288e7048-1964-4683-ab92-3fcacea15656-kube-api-access-fhp9s\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.913002 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.912978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.913135 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.913026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:20.913135 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:20.913116    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 19:25:20.913208 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:20.913168    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm podName:288e7048-1964-4683-ab92-3fcacea15656 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:21.913154416 +0000 UTC m=+41.673909651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s47sv" (UID: "288e7048-1964-4683-ab92-3fcacea15656") : configmap "kube-rbac-proxy" not found
Jul 30 19:25:20.913208 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:20.913121    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:20.913275 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:20.913243    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls podName:288e7048-1964-4683-ab92-3fcacea15656 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:21.913232474 +0000 UTC m=+41.673987711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s47sv" (UID: "288e7048-1964-4683-ab92-3fcacea15656") : secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:20.956308 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:20.956280    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc" event={"ID":"1eee9ece-e46a-432a-aced-5004c9ed090b","Type":"ContainerStarted","Data":"769d3db175e3c779f82c2c990fe06b2113a10656a23470f3384e59d9600f9940"}
Jul 30 19:25:21.920179 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:21.920142    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:21.920528 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:21.920223    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:21.920528 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:21.920292    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:21.920528 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:21.920301    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 19:25:21.920528 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:21.920355    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls podName:288e7048-1964-4683-ab92-3fcacea15656 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:23.920339159 +0000 UTC m=+43.681094416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s47sv" (UID: "288e7048-1964-4683-ab92-3fcacea15656") : secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:21.920528 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:21.920370    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm podName:288e7048-1964-4683-ab92-3fcacea15656 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:23.920364176 +0000 UTC m=+43.681119411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s47sv" (UID: "288e7048-1964-4683-ab92-3fcacea15656") : configmap "kube-rbac-proxy" not found
Jul 30 19:25:22.424283 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.424232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:22.424391 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:22.424375    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:22.424437 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:22.424428    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls podName:c153fb6e-27af-4e8e-a363-1bc0922c458e nodeName:}" failed. No retries permitted until 2026-07-30 19:25:30.424412071 +0000 UTC m=+50.185167326 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls") pod "dns-default-7mqgt" (UID: "c153fb6e-27af-4e8e-a363-1bc0922c458e") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:22.651905 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.651880    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-2xh8n"]
Jul 30 19:25:22.668673 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.668655    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-2xh8n"]
Jul 30 19:25:22.668761 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.668740    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n"
Jul 30 19:25:22.670931 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.670910    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 30 19:25:22.671775 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.671755    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gkrbw\""
Jul 30 19:25:22.671775 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.671769    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 30 19:25:22.671955 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.671830    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 30 19:25:22.671955 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.671847    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 30 19:25:22.824635 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.824616    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4b287_276fa5f2-21c9-42fc-88fc-6f7d3db559c9/dns-node-resolver/0.log"
Jul 30 19:25:22.827839 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.827819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whcbt\" (UniqueName: \"kubernetes.io/projected/b64e3fae-ccb5-40bd-830d-c2fef0e493b0-kube-api-access-whcbt\") pod \"service-ca-98c88fbb5-2xh8n\" (UID: \"b64e3fae-ccb5-40bd-830d-c2fef0e493b0\") " pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n"
Jul 30 19:25:22.827916 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.827852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b64e3fae-ccb5-40bd-830d-c2fef0e493b0-signing-cabundle\") pod \"service-ca-98c88fbb5-2xh8n\" (UID: \"b64e3fae-ccb5-40bd-830d-c2fef0e493b0\") " pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n"
Jul 30 19:25:22.827968 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.827936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b64e3fae-ccb5-40bd-830d-c2fef0e493b0-signing-key\") pod \"service-ca-98c88fbb5-2xh8n\" (UID: \"b64e3fae-ccb5-40bd-830d-c2fef0e493b0\") " pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n"
Jul 30 19:25:22.928673 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.928629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whcbt\" (UniqueName: \"kubernetes.io/projected/b64e3fae-ccb5-40bd-830d-c2fef0e493b0-kube-api-access-whcbt\") pod \"service-ca-98c88fbb5-2xh8n\" (UID: \"b64e3fae-ccb5-40bd-830d-c2fef0e493b0\") " pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n"
Jul 30 19:25:22.928673 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.928656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b64e3fae-ccb5-40bd-830d-c2fef0e493b0-signing-cabundle\") pod \"service-ca-98c88fbb5-2xh8n\" (UID: \"b64e3fae-ccb5-40bd-830d-c2fef0e493b0\") " pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n"
Jul 30 19:25:22.929031 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.928686    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b64e3fae-ccb5-40bd-830d-c2fef0e493b0-signing-key\") pod \"service-ca-98c88fbb5-2xh8n\" (UID: \"b64e3fae-ccb5-40bd-830d-c2fef0e493b0\") " pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n"
Jul 30 19:25:22.929287 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.929271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b64e3fae-ccb5-40bd-830d-c2fef0e493b0-signing-cabundle\") pod \"service-ca-98c88fbb5-2xh8n\" (UID: \"b64e3fae-ccb5-40bd-830d-c2fef0e493b0\") " pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n"
Jul 30 19:25:22.930936 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.930917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b64e3fae-ccb5-40bd-830d-c2fef0e493b0-signing-key\") pod \"service-ca-98c88fbb5-2xh8n\" (UID: \"b64e3fae-ccb5-40bd-830d-c2fef0e493b0\") " pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n"
Jul 30 19:25:22.936222 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.936199    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whcbt\" (UniqueName: \"kubernetes.io/projected/b64e3fae-ccb5-40bd-830d-c2fef0e493b0-kube-api-access-whcbt\") pod \"service-ca-98c88fbb5-2xh8n\" (UID: \"b64e3fae-ccb5-40bd-830d-c2fef0e493b0\") " pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n"
Jul 30 19:25:22.963177 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.963152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc" event={"ID":"1eee9ece-e46a-432a-aced-5004c9ed090b","Type":"ContainerStarted","Data":"e1d993b29c8bc86cf4b0270e1c0ee1bbb362da3c640e4291a6de1a51390bb45c"}
Jul 30 19:25:22.963274 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.963202    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc" event={"ID":"1eee9ece-e46a-432a-aced-5004c9ed090b","Type":"ContainerStarted","Data":"4e190713fb1b794a53fdb33a64fe8b57320e8938d60a538b479b03a60604d7a0"}
Jul 30 19:25:22.976649 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.976628    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n"
Jul 30 19:25:22.977708 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:22.977667    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-76dtc" podStartSLOduration=2.2948288740000002 podStartE2EDuration="3.977651834s" totalImagesPullingTime="1.68282296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:22.976664689 +0000 UTC m=+42.737419948" watchObservedRunningTime="2026-07-30 19:25:22.977651834 +0000 UTC m=+42.738407095"
Jul 30 19:25:23.108061 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:23.107785    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-2xh8n"]
Jul 30 19:25:23.110353 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:25:23.110328    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64e3fae_ccb5_40bd_830d_c2fef0e493b0.slice/crio-3096e6915d823c940eb8f316ecd818ac7cbefff6b463324d3f03f1390543afd1 WatchSource:0}: Error finding container 3096e6915d823c940eb8f316ecd818ac7cbefff6b463324d3f03f1390543afd1: Status 404 returned error can't find the container with id 3096e6915d823c940eb8f316ecd818ac7cbefff6b463324d3f03f1390543afd1
Jul 30 19:25:23.825525 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:23.825499    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7j96x_3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1/node-ca/0.log"
Jul 30 19:25:23.935131 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:23.935099    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:23.935578 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:23.935191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:23.935578 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:23.935320    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:23.935578 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:23.935381    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls podName:288e7048-1964-4683-ab92-3fcacea15656 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:27.935367287 +0000 UTC m=+47.696122540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s47sv" (UID: "288e7048-1964-4683-ab92-3fcacea15656") : secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:23.935755 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:23.935696    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/288e7048-1964-4683-ab92-3fcacea15656-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:23.967139 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:23.967102    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n" event={"ID":"b64e3fae-ccb5-40bd-830d-c2fef0e493b0","Type":"ContainerStarted","Data":"3096e6915d823c940eb8f316ecd818ac7cbefff6b463324d3f03f1390543afd1"}
Jul 30 19:25:24.625370 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:24.625333    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ftznv_a958f2b4-3331-4143-8794-7297a6e533ec/serve-healthcheck-canary/0.log"
Jul 30 19:25:25.225851 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:25.225825    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-76dtc_1eee9ece-e46a-432a-aced-5004c9ed090b/migrator/0.log"
Jul 30 19:25:25.425601 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:25.425576    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-76dtc_1eee9ece-e46a-432a-aced-5004c9ed090b/graceful-termination/0.log"
Jul 30 19:25:25.973153 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:25.973082    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n" event={"ID":"b64e3fae-ccb5-40bd-830d-c2fef0e493b0","Type":"ContainerStarted","Data":"e3fd0e5a3294560de1efea50b042f9a854c142dc5eeeecc2f40d7456b24a01fd"}
Jul 30 19:25:25.989018 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:25.988975    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-2xh8n" podStartSLOduration=1.419623466 podStartE2EDuration="3.988960057s" totalImagesPullingTime="2.569336591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:25.988691392 +0000 UTC m=+45.749446651" watchObservedRunningTime="2026-07-30 19:25:25.988960057 +0000 UTC m=+45.749715308"
Jul 30 19:25:27.962680 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:27.962641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:27.963161 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:27.962795    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:27.963161 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:27.962862    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls podName:288e7048-1964-4683-ab92-3fcacea15656 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:35.962841807 +0000 UTC m=+55.723597059 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s47sv" (UID: "288e7048-1964-4683-ab92-3fcacea15656") : secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:30.481999 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:30.481967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:30.482543 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:30.482147    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:30.482543 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:25:30.482218    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls podName:c153fb6e-27af-4e8e-a363-1bc0922c458e nodeName:}" failed. No retries permitted until 2026-07-30 19:25:46.482197768 +0000 UTC m=+66.242953022 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls") pod "dns-default-7mqgt" (UID: "c153fb6e-27af-4e8e-a363-1bc0922c458e") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:36.016741 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:36.016707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:36.018889 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:36.018871    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/288e7048-1964-4683-ab92-3fcacea15656-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s47sv\" (UID: \"288e7048-1964-4683-ab92-3fcacea15656\") " pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:36.080013 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:36.079990    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s47sv"
Jul 30 19:25:36.198511 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:36.198481    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s47sv"]
Jul 30 19:25:36.201861 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:25:36.201833    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod288e7048_1964_4683_ab92_3fcacea15656.slice/crio-928c4d9c8b6093c2f41f25f51a123af6f05f819ca1355758fcd4a4fcc9c278aa WatchSource:0}: Error finding container 928c4d9c8b6093c2f41f25f51a123af6f05f819ca1355758fcd4a4fcc9c278aa: Status 404 returned error can't find the container with id 928c4d9c8b6093c2f41f25f51a123af6f05f819ca1355758fcd4a4fcc9c278aa
Jul 30 19:25:37.002239 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:37.002203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s47sv" event={"ID":"288e7048-1964-4683-ab92-3fcacea15656","Type":"ContainerStarted","Data":"0eaf0a7af2fc535b42858ef1af96d662555ebb02976bdc2741955f40cef25af3"}
Jul 30 19:25:37.002332 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:37.002301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s47sv" event={"ID":"288e7048-1964-4683-ab92-3fcacea15656","Type":"ContainerStarted","Data":"8bb53b44ac627a6a438f3e0f1f2db98241c28ab447a4832f643a5eec8fc80ba4"}
Jul 30 19:25:37.002332 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:37.002330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s47sv" event={"ID":"288e7048-1964-4683-ab92-3fcacea15656","Type":"ContainerStarted","Data":"928c4d9c8b6093c2f41f25f51a123af6f05f819ca1355758fcd4a4fcc9c278aa"}
Jul 30 19:25:39.010716 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:39.010682    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s47sv" event={"ID":"288e7048-1964-4683-ab92-3fcacea15656","Type":"ContainerStarted","Data":"a00a2c7fb9750ae25c346cdb5b76cb6d90fede4f24107f113587305ae9f3f9cb"}
Jul 30 19:25:39.027706 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:39.027663    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s47sv" podStartSLOduration=16.887526284 podStartE2EDuration="19.02764968s" totalImagesPullingTime="2.140123396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:25:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:39.026893392 +0000 UTC m=+58.787648660" watchObservedRunningTime="2026-07-30 19:25:39.02764968 +0000 UTC m=+58.788404938"
Jul 30 19:25:40.921128 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:40.921101    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2r5sc"
Jul 30 19:25:45.005469 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.005436    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh"]
Jul 30 19:25:45.008300 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.008267    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh"
Jul 30 19:25:45.009058 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.009029    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"]
Jul 30 19:25:45.010985 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.010966    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.011793 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.011764    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6tqjf\""
Jul 30 19:25:45.011892 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.011794    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 19:25:45.011892 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.011803    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 19:25:45.011892 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.011804    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 30 19:25:45.011892 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.011764    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 19:25:45.013105 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.013086    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 30 19:25:45.014063 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.014040    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 30 19:25:45.014471 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.014456    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 30 19:25:45.014645 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.014623    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 30 19:25:45.018161 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.018138    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh"]
Jul 30 19:25:45.026382 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.026362    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"]
Jul 30 19:25:45.080419 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.080396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.080564 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.080426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd0e71b6-714b-4aa4-b797-b6433bc8722d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67c6d79dcd-np7wh\" (UID: \"bd0e71b6-714b-4aa4-b797-b6433bc8722d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh"
Jul 30 19:25:45.080564 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.080450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg4rp\" (UniqueName: \"kubernetes.io/projected/bd0e71b6-714b-4aa4-b797-b6433bc8722d-kube-api-access-lg4rp\") pod \"managed-serviceaccount-addon-agent-67c6d79dcd-np7wh\" (UID: \"bd0e71b6-714b-4aa4-b797-b6433bc8722d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh"
Jul 30 19:25:45.080564 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.080485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-hub\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.080564 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.080512    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-additional-service-ca\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.080564 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.080562    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.080723 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.080595    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-ca\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.080723 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.080619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljpsw\" (UniqueName: \"kubernetes.io/projected/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-kube-api-access-ljpsw\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.181589 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.181565    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-ca\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.181693 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.181592    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljpsw\" (UniqueName: \"kubernetes.io/projected/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-kube-api-access-ljpsw\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.181693 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.181626    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.181839 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.181821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd0e71b6-714b-4aa4-b797-b6433bc8722d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67c6d79dcd-np7wh\" (UID: \"bd0e71b6-714b-4aa4-b797-b6433bc8722d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh"
Jul 30 19:25:45.181899 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.181874    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg4rp\" (UniqueName: \"kubernetes.io/projected/bd0e71b6-714b-4aa4-b797-b6433bc8722d-kube-api-access-lg4rp\") pod \"managed-serviceaccount-addon-agent-67c6d79dcd-np7wh\" (UID: \"bd0e71b6-714b-4aa4-b797-b6433bc8722d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh"
Jul 30 19:25:45.181956 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.181900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-hub\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.181956 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.181926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-additional-service-ca\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.182053 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.181984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.182672 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.182649    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-additional-service-ca\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.184326 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.184286    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.184582 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.184561    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-ca\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.184650 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.184583    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.184726 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.184712    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-hub\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.184780 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.184713    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd0e71b6-714b-4aa4-b797-b6433bc8722d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67c6d79dcd-np7wh\" (UID: \"bd0e71b6-714b-4aa4-b797-b6433bc8722d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh"
Jul 30 19:25:45.196101 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.196057    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg4rp\" (UniqueName: \"kubernetes.io/projected/bd0e71b6-714b-4aa4-b797-b6433bc8722d-kube-api-access-lg4rp\") pod \"managed-serviceaccount-addon-agent-67c6d79dcd-np7wh\" (UID: \"bd0e71b6-714b-4aa4-b797-b6433bc8722d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh"
Jul 30 19:25:45.196217 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.196201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljpsw\" (UniqueName: \"kubernetes.io/projected/ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad-kube-api-access-ljpsw\") pod \"cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf\" (UID: \"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.325347 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.325330    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh"
Jul 30 19:25:45.331041 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.331021    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"
Jul 30 19:25:45.449705 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.449616    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh"]
Jul 30 19:25:45.452156 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:25:45.452124    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd0e71b6_714b_4aa4_b797_b6433bc8722d.slice/crio-1df717d7245369073aa364c43d43cb5c078f4856f39d8a0214498bdf7d0ede45 WatchSource:0}: Error finding container 1df717d7245369073aa364c43d43cb5c078f4856f39d8a0214498bdf7d0ede45: Status 404 returned error can't find the container with id 1df717d7245369073aa364c43d43cb5c078f4856f39d8a0214498bdf7d0ede45
Jul 30 19:25:45.465898 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:45.465874    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf"]
Jul 30 19:25:45.469010 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:25:45.468984    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee3e39ce_7f1a_4866_bf4e_c94d0e1e07ad.slice/crio-e6ad1ad1d0ba64f164504dc287651c14c7158e6d19afe24283e085ac6d7d0fe7 WatchSource:0}: Error finding container e6ad1ad1d0ba64f164504dc287651c14c7158e6d19afe24283e085ac6d7d0fe7: Status 404 returned error can't find the container with id e6ad1ad1d0ba64f164504dc287651c14c7158e6d19afe24283e085ac6d7d0fe7
Jul 30 19:25:46.032267 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.032240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf" event={"ID":"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad","Type":"ContainerStarted","Data":"e6ad1ad1d0ba64f164504dc287651c14c7158e6d19afe24283e085ac6d7d0fe7"}
Jul 30 19:25:46.033310 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.033280    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh" event={"ID":"bd0e71b6-714b-4aa4-b797-b6433bc8722d","Type":"ContainerStarted","Data":"1df717d7245369073aa364c43d43cb5c078f4856f39d8a0214498bdf7d0ede45"}
Jul 30 19:25:46.389824 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.389797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:46.392329 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.392312    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 19:25:46.402188 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.402168    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3fe769f8-bc2c-466c-be78-95ef7f74263c-original-pull-secret\") pod \"global-pull-secret-syncer-7dmqn\" (UID: \"3fe769f8-bc2c-466c-be78-95ef7f74263c\") " pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:46.490632 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.490605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:46.490731 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.490642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:46.492812 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.492791    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c153fb6e-27af-4e8e-a363-1bc0922c458e-metrics-tls\") pod \"dns-default-7mqgt\" (UID: \"c153fb6e-27af-4e8e-a363-1bc0922c458e\") " pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:46.493080 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.493046    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 19:25:46.503005 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.502984    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99fc519b-8e3c-4564-a65a-f81a314974f9-metrics-certs\") pod \"network-metrics-daemon-26gjf\" (UID: \"99fc519b-8e3c-4564-a65a-f81a314974f9\") " pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:46.591248 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.591222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsx9g\" (UniqueName: \"kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g\") pod \"network-check-target-8ml9s\" (UID: \"6e6a374a-1f04-4351-bb56-8e329d65bc25\") " pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:46.593233 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.593212    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7dmqn"
Jul 30 19:25:46.594838 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.594815    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 19:25:46.601694 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.601653    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bgfkv\""
Jul 30 19:25:46.603648 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.603627    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 19:25:46.609708 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.609688    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26gjf"
Jul 30 19:25:46.614752 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.614726    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsx9g\" (UniqueName: \"kubernetes.io/projected/6e6a374a-1f04-4351-bb56-8e329d65bc25-kube-api-access-lsx9g\") pod \"network-check-target-8ml9s\" (UID: \"6e6a374a-1f04-4351-bb56-8e329d65bc25\") " pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:46.769500 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.769032    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7dmqn"]
Jul 30 19:25:46.771699 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.771481    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9htd7\""
Jul 30 19:25:46.779810 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.779691    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:46.781235 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:25:46.781206    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe769f8_bc2c_466c_be78_95ef7f74263c.slice/crio-32ab776f565ecfb29d21bc6daaa090e7dff4a20b60eb0ba133d2bd7dea40c211 WatchSource:0}: Error finding container 32ab776f565ecfb29d21bc6daaa090e7dff4a20b60eb0ba133d2bd7dea40c211: Status 404 returned error can't find the container with id 32ab776f565ecfb29d21bc6daaa090e7dff4a20b60eb0ba133d2bd7dea40c211
Jul 30 19:25:46.815747 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.815718    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-26gjf"]
Jul 30 19:25:46.820462 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:25:46.820432    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99fc519b_8e3c_4564_a65a_f81a314974f9.slice/crio-64a9ee9a5eb19ccd66768a01ac01f82ccf4692e2b7eebe2ad31050fca9888482 WatchSource:0}: Error finding container 64a9ee9a5eb19ccd66768a01ac01f82ccf4692e2b7eebe2ad31050fca9888482: Status 404 returned error can't find the container with id 64a9ee9a5eb19ccd66768a01ac01f82ccf4692e2b7eebe2ad31050fca9888482
Jul 30 19:25:46.908265 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.908206    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-82x98\""
Jul 30 19:25:46.916326 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.916298    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:46.982064 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:46.981980    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7mqgt"]
Jul 30 19:25:46.990346 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:25:46.990313    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc153fb6e_27af_4e8e_a363_1bc0922c458e.slice/crio-6ff1be8e4dc42ff465198d8929436740ec4a4d7054085e9054cf6fc7aa474c28 WatchSource:0}: Error finding container 6ff1be8e4dc42ff465198d8929436740ec4a4d7054085e9054cf6fc7aa474c28: Status 404 returned error can't find the container with id 6ff1be8e4dc42ff465198d8929436740ec4a4d7054085e9054cf6fc7aa474c28
Jul 30 19:25:47.043229 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:47.043047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7mqgt" event={"ID":"c153fb6e-27af-4e8e-a363-1bc0922c458e","Type":"ContainerStarted","Data":"6ff1be8e4dc42ff465198d8929436740ec4a4d7054085e9054cf6fc7aa474c28"}
Jul 30 19:25:47.046283 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:47.046199    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-26gjf" event={"ID":"99fc519b-8e3c-4564-a65a-f81a314974f9","Type":"ContainerStarted","Data":"64a9ee9a5eb19ccd66768a01ac01f82ccf4692e2b7eebe2ad31050fca9888482"}
Jul 30 19:25:47.049084 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:47.049005    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7dmqn" event={"ID":"3fe769f8-bc2c-466c-be78-95ef7f74263c","Type":"ContainerStarted","Data":"32ab776f565ecfb29d21bc6daaa090e7dff4a20b60eb0ba133d2bd7dea40c211"}
Jul 30 19:25:47.118445 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:47.117990    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8ml9s"]
Jul 30 19:25:47.121393 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:25:47.121365    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e6a374a_1f04_4351_bb56_8e329d65bc25.slice/crio-e14ffa60c9e9eedd3422f430da253d41a4ef8e3695ca28039ced752f378617b7 WatchSource:0}: Error finding container e14ffa60c9e9eedd3422f430da253d41a4ef8e3695ca28039ced752f378617b7: Status 404 returned error can't find the container with id e14ffa60c9e9eedd3422f430da253d41a4ef8e3695ca28039ced752f378617b7
Jul 30 19:25:48.057927 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:48.057828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8ml9s" event={"ID":"6e6a374a-1f04-4351-bb56-8e329d65bc25","Type":"ContainerStarted","Data":"e14ffa60c9e9eedd3422f430da253d41a4ef8e3695ca28039ced752f378617b7"}
Jul 30 19:25:55.086521 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.086485    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7mqgt" event={"ID":"c153fb6e-27af-4e8e-a363-1bc0922c458e","Type":"ContainerStarted","Data":"0d23adfd5abd62a7db6d75c048a6ef6da7d6dc372cd8647242678a69e90c7ab2"}
Jul 30 19:25:55.086948 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.086541    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7mqgt" event={"ID":"c153fb6e-27af-4e8e-a363-1bc0922c458e","Type":"ContainerStarted","Data":"6effde8d36d67c397869c837ff94c94e4d042af7b8d560b7f684b3820e4d1aca"}
Jul 30 19:25:55.086948 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.086568    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:25:55.088316 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.088288    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-26gjf" event={"ID":"99fc519b-8e3c-4564-a65a-f81a314974f9","Type":"ContainerStarted","Data":"0fea14b4298936ca3fc06a12312e6bf60ea2909a4666abafee30beffc592823e"}
Jul 30 19:25:55.088422 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.088346    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-26gjf" event={"ID":"99fc519b-8e3c-4564-a65a-f81a314974f9","Type":"ContainerStarted","Data":"106130d256d04b22aeafbefa39f760c54099012ed358e9405213def0b917972c"}
Jul 30 19:25:55.089636 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.089613    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7dmqn" event={"ID":"3fe769f8-bc2c-466c-be78-95ef7f74263c","Type":"ContainerStarted","Data":"a236be46d50f0b4acd1bd309d299af67b28bc960c8516960920406c012b76e90"}
Jul 30 19:25:55.091727 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.091702    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf" event={"ID":"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad","Type":"ContainerStarted","Data":"183e2a96f181abb23ab4061a86a523e30ec3aebdc7d9d1bbadb14a60fdeabb93"}
Jul 30 19:25:55.091843 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.091745    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf" event={"ID":"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad","Type":"ContainerStarted","Data":"7e29e004e0db755381d60acc52a6466c1d82d7576ec010036655d382ea0fa789"}
Jul 30 19:25:55.091843 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.091764    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf" event={"ID":"ee3e39ce-7f1a-4866-bf4e-c94d0e1e07ad","Type":"ContainerStarted","Data":"e06ba630300c9a88ed74536421980a4b3fdcfc7bde62bfa26f282648839790c1"}
Jul 30 19:25:55.093297 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.093272    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8ml9s" event={"ID":"6e6a374a-1f04-4351-bb56-8e329d65bc25","Type":"ContainerStarted","Data":"5b99eacc65babe749a204055387f20cb7e13d66fac3a28914b916112fded6270"}
Jul 30 19:25:55.093394 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.093356    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:25:55.094637 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.094616    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh" event={"ID":"bd0e71b6-714b-4aa4-b797-b6433bc8722d","Type":"ContainerStarted","Data":"2dbcd3ffeafb46245155293f62f467b69cea23ab3e2179d28974011ae23af3cf"}
Jul 30 19:25:55.105756 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.105718    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7mqgt" podStartSLOduration=33.471476985 podStartE2EDuration="41.105693493s" totalImagesPullingTime="7.634216508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:55.10468905 +0000 UTC m=+74.865444307" watchObservedRunningTime="2026-07-30 19:25:55.105693493 +0000 UTC m=+74.866448732"
Jul 30 19:25:55.119247 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.119177    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67c6d79dcd-np7wh" podStartSLOduration=1.9507774530000002 podStartE2EDuration="11.119163932s" totalImagesPullingTime="9.168386479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:55.118365964 +0000 UTC m=+74.879121224" watchObservedRunningTime="2026-07-30 19:25:55.119163932 +0000 UTC m=+74.879919186"
Jul 30 19:25:55.137317 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.137274    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65fb4dfdf5-f4mzf" podStartSLOduration=1.960561714 podStartE2EDuration="11.137262189s" totalImagesPullingTime="9.176700475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:55.13651291 +0000 UTC m=+74.897268179" watchObservedRunningTime="2026-07-30 19:25:55.137262189 +0000 UTC m=+74.898017447"
Jul 30 19:25:55.152128 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.151964    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-26gjf" podStartSLOduration=67.354015221 podStartE2EDuration="1m15.151949027s" totalImagesPullingTime="7.797933806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:55.151279214 +0000 UTC m=+74.912034475" watchObservedRunningTime="2026-07-30 19:25:55.151949027 +0000 UTC m=+74.912704286"
Jul 30 19:25:55.165955 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.165919    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7dmqn" podStartSLOduration=67.31766085 podStartE2EDuration="1m15.165907712s" totalImagesPullingTime="7.848246862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:55.16521302 +0000 UTC m=+74.925968279" watchObservedRunningTime="2026-07-30 19:25:55.165907712 +0000 UTC m=+74.926662970"
Jul 30 19:25:55.179407 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:25:55.179353    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8ml9s" podStartSLOduration=67.651610465 podStartE2EDuration="1m15.179338464s" totalImagesPullingTime="7.527727999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:55.179046151 +0000 UTC m=+74.939801409" watchObservedRunningTime="2026-07-30 19:25:55.179338464 +0000 UTC m=+74.940093723"
Jul 30 19:26:05.103281 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:05.103178    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7mqgt"
Jul 30 19:26:07.346423 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.346388    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6f8cs"]
Jul 30 19:26:07.351155 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.351135    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.353343 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.353309    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 19:26:07.353478 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.353360    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 19:26:07.353478 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.353463    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-twtn8\""
Jul 30 19:26:07.353790 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.353767    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 19:26:07.353892 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.353831    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 19:26:07.354259 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.354239    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 19:26:07.354358 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.354321    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 19:26:07.426524 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.426502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-sys\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.426636 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.426543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.426636 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.426570    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-tls\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.426737 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.426637    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9zz6\" (UniqueName: \"kubernetes.io/projected/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-kube-api-access-s9zz6\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.426785 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.426747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.426825 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.426804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-metrics-client-ca\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.426869 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.426837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-wtmp\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.426910 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.426876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-root\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.426910 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.426903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-textfile\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527391 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-root\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527491 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-textfile\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527491 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527430    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-sys\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527491 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527640 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527484    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-root\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527640 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-sys\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527640 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-tls\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527640 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9zz6\" (UniqueName: \"kubernetes.io/projected/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-kube-api-access-s9zz6\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527640 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527899 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-metrics-client-ca\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527899 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-wtmp\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527899 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527722    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-textfile\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.527899 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.527805    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-wtmp\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.528118 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.528064    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-accelerators-collector-config\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.528297 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.528268    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-metrics-client-ca\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.529872 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.529845    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-tls\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.529944 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.529934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.535105 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.535085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9zz6\" (UniqueName: \"kubernetes.io/projected/7b76526d-44ac-43f8-acfa-b1feb0ea88ea-kube-api-access-s9zz6\") pod \"node-exporter-6f8cs\" (UID: \"7b76526d-44ac-43f8-acfa-b1feb0ea88ea\") " pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.663222 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:07.663150    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6f8cs"
Jul 30 19:26:07.674460 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:26:07.674437    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b76526d_44ac_43f8_acfa_b1feb0ea88ea.slice/crio-5794fbc890ea424a1c586c4f0f8d1f001a2e0801dd1957c164e60c45a62cfbae WatchSource:0}: Error finding container 5794fbc890ea424a1c586c4f0f8d1f001a2e0801dd1957c164e60c45a62cfbae: Status 404 returned error can't find the container with id 5794fbc890ea424a1c586c4f0f8d1f001a2e0801dd1957c164e60c45a62cfbae
Jul 30 19:26:08.137774 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:08.137750    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f8cs" event={"ID":"7b76526d-44ac-43f8-acfa-b1feb0ea88ea","Type":"ContainerStarted","Data":"5794fbc890ea424a1c586c4f0f8d1f001a2e0801dd1957c164e60c45a62cfbae"}
Jul 30 19:26:09.141947 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:09.141923    2577 generic.go:356] "Generic (PLEG): container finished" podID="7b76526d-44ac-43f8-acfa-b1feb0ea88ea" containerID="0c2f46e3a23d81304d60d7606f537eb3d2a3d428e043a9fc156bf9518e20eb97" exitCode=0
Jul 30 19:26:09.142306 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:09.142002    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f8cs" event={"ID":"7b76526d-44ac-43f8-acfa-b1feb0ea88ea","Type":"ContainerDied","Data":"0c2f46e3a23d81304d60d7606f537eb3d2a3d428e043a9fc156bf9518e20eb97"}
Jul 30 19:26:10.147208 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:10.147170    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f8cs" event={"ID":"7b76526d-44ac-43f8-acfa-b1feb0ea88ea","Type":"ContainerStarted","Data":"8bf7b99e6b46761ffa3baa690142492eba5511196ee48b0f3b62933f2df27281"}
Jul 30 19:26:10.147592 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:10.147265    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6f8cs" event={"ID":"7b76526d-44ac-43f8-acfa-b1feb0ea88ea","Type":"ContainerStarted","Data":"58276d779be1db75db8b8b24aa9991348c55508b18202dc2a78809b711c6af1a"}
Jul 30 19:26:26.103649 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:26.103618    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8ml9s"
Jul 30 19:26:26.119869 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:26:26.119811    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6f8cs" podStartSLOduration=17.785519693 podStartE2EDuration="19.119796927s" totalImagesPullingTime="1.273389243s" totalInitContainerRuntime="60.887991ms" isStatefulPod=true podCreationTimestamp="2026-07-30 19:26:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:26:10.169060107 +0000 UTC m=+89.929815364" watchObservedRunningTime="2026-07-30 19:26:26.119796927 +0000 UTC m=+105.880552255"
Jul 30 19:29:31.160064 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.160038    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c7qgw"]
Jul 30 19:29:31.161857 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.161843    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c7qgw"
Jul 30 19:29:31.164329 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.164301    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-kbd2x\""
Jul 30 19:29:31.164437 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.164311    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 30 19:29:31.165009 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.164993    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 30 19:29:31.171991 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.171967    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c7qgw"]
Jul 30 19:29:31.300458 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.300434    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh5pf\" (UniqueName: \"kubernetes.io/projected/b02a8562-6f20-4cfc-9a38-9875c0409c6e-kube-api-access-dh5pf\") pod \"cert-manager-55b4d954d5-c7qgw\" (UID: \"b02a8562-6f20-4cfc-9a38-9875c0409c6e\") " pod="cert-manager/cert-manager-55b4d954d5-c7qgw"
Jul 30 19:29:31.300552 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.300486    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b02a8562-6f20-4cfc-9a38-9875c0409c6e-bound-sa-token\") pod \"cert-manager-55b4d954d5-c7qgw\" (UID: \"b02a8562-6f20-4cfc-9a38-9875c0409c6e\") " pod="cert-manager/cert-manager-55b4d954d5-c7qgw"
Jul 30 19:29:31.401103 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.401084    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b02a8562-6f20-4cfc-9a38-9875c0409c6e-bound-sa-token\") pod \"cert-manager-55b4d954d5-c7qgw\" (UID: \"b02a8562-6f20-4cfc-9a38-9875c0409c6e\") " pod="cert-manager/cert-manager-55b4d954d5-c7qgw"
Jul 30 19:29:31.401189 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.401111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh5pf\" (UniqueName: \"kubernetes.io/projected/b02a8562-6f20-4cfc-9a38-9875c0409c6e-kube-api-access-dh5pf\") pod \"cert-manager-55b4d954d5-c7qgw\" (UID: \"b02a8562-6f20-4cfc-9a38-9875c0409c6e\") " pod="cert-manager/cert-manager-55b4d954d5-c7qgw"
Jul 30 19:29:31.412351 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.412294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b02a8562-6f20-4cfc-9a38-9875c0409c6e-bound-sa-token\") pod \"cert-manager-55b4d954d5-c7qgw\" (UID: \"b02a8562-6f20-4cfc-9a38-9875c0409c6e\") " pod="cert-manager/cert-manager-55b4d954d5-c7qgw"
Jul 30 19:29:31.412351 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.412345    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh5pf\" (UniqueName: \"kubernetes.io/projected/b02a8562-6f20-4cfc-9a38-9875c0409c6e-kube-api-access-dh5pf\") pod \"cert-manager-55b4d954d5-c7qgw\" (UID: \"b02a8562-6f20-4cfc-9a38-9875c0409c6e\") " pod="cert-manager/cert-manager-55b4d954d5-c7qgw"
Jul 30 19:29:31.470235 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.470215    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c7qgw"
Jul 30 19:29:31.587730 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.587699    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c7qgw"]
Jul 30 19:29:31.590782 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:29:31.590756    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb02a8562_6f20_4cfc_9a38_9875c0409c6e.slice/crio-3676f3e6f1ece40b7d5c966ee8a347c0a21da7822d773f76101e11bc8d419261 WatchSource:0}: Error finding container 3676f3e6f1ece40b7d5c966ee8a347c0a21da7822d773f76101e11bc8d419261: Status 404 returned error can't find the container with id 3676f3e6f1ece40b7d5c966ee8a347c0a21da7822d773f76101e11bc8d419261
Jul 30 19:29:31.756694 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:31.756632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c7qgw" event={"ID":"b02a8562-6f20-4cfc-9a38-9875c0409c6e","Type":"ContainerStarted","Data":"3676f3e6f1ece40b7d5c966ee8a347c0a21da7822d773f76101e11bc8d419261"}
Jul 30 19:29:35.771454 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:35.771418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c7qgw" event={"ID":"b02a8562-6f20-4cfc-9a38-9875c0409c6e","Type":"ContainerStarted","Data":"fbaceb25f9a02bb5371c25ac8838c2a2ea8764e8f83b7926377f49a2c63c6023"}
Jul 30 19:29:35.787789 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:35.787741    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-c7qgw" podStartSLOduration=1.580509862 podStartE2EDuration="4.787727098s" totalImagesPullingTime="3.207217236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:29:35.786883213 +0000 UTC m=+295.547638472" watchObservedRunningTime="2026-07-30 19:29:35.787727098 +0000 UTC m=+295.548482356"
Jul 30 19:29:40.619263 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:40.619237    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 19:29:54.622807 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.622770    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"]
Jul 30 19:29:54.626441 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.626430    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.633461 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.633443    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4kpj4\""
Jul 30 19:29:54.633923 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.633905    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 30 19:29:54.634000 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.633982    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 30 19:29:54.634038 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.633995    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 30 19:29:54.634288 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.634270    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:29:54.634333 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.634289    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 30 19:29:54.654110 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.654086    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"]
Jul 30 19:29:54.742010 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.741989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e086d1f-d826-4bd4-bb39-2930137f9d87-metrics-cert\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.742154 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.742027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e086d1f-d826-4bd4-bb39-2930137f9d87-cert\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.742154 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.742058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0e086d1f-d826-4bd4-bb39-2930137f9d87-manager-config\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.742154 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.742095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfrx4\" (UniqueName: \"kubernetes.io/projected/0e086d1f-d826-4bd4-bb39-2930137f9d87-kube-api-access-jfrx4\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.842804 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.842777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e086d1f-d826-4bd4-bb39-2930137f9d87-cert\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.842911 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.842818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0e086d1f-d826-4bd4-bb39-2930137f9d87-manager-config\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.842911 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.842846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfrx4\" (UniqueName: \"kubernetes.io/projected/0e086d1f-d826-4bd4-bb39-2930137f9d87-kube-api-access-jfrx4\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.842911 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.842881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e086d1f-d826-4bd4-bb39-2930137f9d87-metrics-cert\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.843480 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.843462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0e086d1f-d826-4bd4-bb39-2930137f9d87-manager-config\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.845588 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.845569    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e086d1f-d826-4bd4-bb39-2930137f9d87-cert\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.845683 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.845633    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e086d1f-d826-4bd4-bb39-2930137f9d87-metrics-cert\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.853999 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.853978    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfrx4\" (UniqueName: \"kubernetes.io/projected/0e086d1f-d826-4bd4-bb39-2930137f9d87-kube-api-access-jfrx4\") pod \"lws-controller-manager-659fd749fb-6fq69\" (UID: \"0e086d1f-d826-4bd4-bb39-2930137f9d87\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:54.935372 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:54.935314    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:55.074763 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.074740    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"]
Jul 30 19:29:55.076791 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:29:55.076763    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e086d1f_d826_4bd4_bb39_2930137f9d87.slice/crio-21b9ef994f84a2e6cfb21cd3b34c49f8648d1851eeaed37aa4bd8a548b8f3fc3 WatchSource:0}: Error finding container 21b9ef994f84a2e6cfb21cd3b34c49f8648d1851eeaed37aa4bd8a548b8f3fc3: Status 404 returned error can't find the container with id 21b9ef994f84a2e6cfb21cd3b34c49f8648d1851eeaed37aa4bd8a548b8f3fc3
Jul 30 19:29:55.078760 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.078745    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 19:29:55.111580 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.111553    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"]
Jul 30 19:29:55.114376 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.114360    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.118998 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.118978    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 19:29:55.119238 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.119225    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 19:29:55.119478 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.119455    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-29w77\""
Jul 30 19:29:55.119914 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.119895    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 19:29:55.120130 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.120115    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 19:29:55.141238 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.140150    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"]
Jul 30 19:29:55.246094 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.246025    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdpg2\" (UniqueName: \"kubernetes.io/projected/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-kube-api-access-zdpg2\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.246094 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.246059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.246223 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.246100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-tmp\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.246223 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.246189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-manifests\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.246223 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.246219    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.346437 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.346415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-manifests\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.346541 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.346443    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.346541 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.346462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdpg2\" (UniqueName: \"kubernetes.io/projected/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-kube-api-access-zdpg2\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.346541 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.346481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.346541 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.346506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-tmp\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.346967 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.346941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-manifests\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.348609 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.348590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-tmp\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.348964 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.348944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.349009 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.348953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.354217 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.354195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdpg2\" (UniqueName: \"kubernetes.io/projected/647aaeb7-63c8-4ffc-ab6e-ea808a667b72-kube-api-access-zdpg2\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6\" (UID: \"647aaeb7-63c8-4ffc-ab6e-ea808a667b72\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.424320 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.424296    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:29:55.553762 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.553714    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"]
Jul 30 19:29:55.556380 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:29:55.556357    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod647aaeb7_63c8_4ffc_ab6e_ea808a667b72.slice/crio-a69c032ed4a3ac3f738e5c221460ffdba933b37a74c76f9a9a2e2521c16bbb8e WatchSource:0}: Error finding container a69c032ed4a3ac3f738e5c221460ffdba933b37a74c76f9a9a2e2521c16bbb8e: Status 404 returned error can't find the container with id a69c032ed4a3ac3f738e5c221460ffdba933b37a74c76f9a9a2e2521c16bbb8e
Jul 30 19:29:55.835964 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.835912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69" event={"ID":"0e086d1f-d826-4bd4-bb39-2930137f9d87","Type":"ContainerStarted","Data":"21b9ef994f84a2e6cfb21cd3b34c49f8648d1851eeaed37aa4bd8a548b8f3fc3"}
Jul 30 19:29:55.837436 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:55.837396    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6" event={"ID":"647aaeb7-63c8-4ffc-ab6e-ea808a667b72","Type":"ContainerStarted","Data":"a69c032ed4a3ac3f738e5c221460ffdba933b37a74c76f9a9a2e2521c16bbb8e"}
Jul 30 19:29:59.854061 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:59.854027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69" event={"ID":"0e086d1f-d826-4bd4-bb39-2930137f9d87","Type":"ContainerStarted","Data":"89f49e2f912924d39f247d111dd54d782078363b6450efc1adb7a5813da6c470"}
Jul 30 19:29:59.854506 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:59.854175    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:29:59.855620 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:59.855594    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6" event={"ID":"647aaeb7-63c8-4ffc-ab6e-ea808a667b72","Type":"ContainerStarted","Data":"8c43e300398f448caf92cda986e12716b449e184a4aec60b44744ca16255790e"}
Jul 30 19:29:59.870671 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:29:59.870630    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69" podStartSLOduration=1.801825096 podStartE2EDuration="5.870615397s" totalImagesPullingTime="4.068790301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:29:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:29:59.869909596 +0000 UTC m=+319.630664879" watchObservedRunningTime="2026-07-30 19:29:59.870615397 +0000 UTC m=+319.631370655"
Jul 30 19:30:00.860425 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:00.860382    2577 generic.go:356] "Generic (PLEG): container finished" podID="647aaeb7-63c8-4ffc-ab6e-ea808a667b72" containerID="8c43e300398f448caf92cda986e12716b449e184a4aec60b44744ca16255790e" exitCode=0
Jul 30 19:30:00.860794 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:00.860462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6" event={"ID":"647aaeb7-63c8-4ffc-ab6e-ea808a667b72","Type":"ContainerDied","Data":"8c43e300398f448caf92cda986e12716b449e184a4aec60b44744ca16255790e"}
Jul 30 19:30:01.866213 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:01.866179    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6" event={"ID":"647aaeb7-63c8-4ffc-ab6e-ea808a667b72","Type":"ContainerStarted","Data":"f04594623cf9b5f439aad0a70963258407e2d97dad92be767a6ce31b42a907a5"}
Jul 30 19:30:01.866690 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:01.866309    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:30:01.889040 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:01.888994    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6" podStartSLOduration=1.810576249 podStartE2EDuration="6.888978769s" totalImagesPullingTime="3.996769053s" totalInitContainerRuntime="1.081633467s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:29:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:30:01.887031289 +0000 UTC m=+321.647786546" watchObservedRunningTime="2026-07-30 19:30:01.888978769 +0000 UTC m=+321.649734018"
Jul 30 19:30:10.862720 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:10.862695    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-6fq69"
Jul 30 19:30:12.871484 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:12.871457    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6"
Jul 30 19:30:25.400606 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.400563    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"]
Jul 30 19:30:25.407308 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.407274    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:25.411010 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.410991    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 19:30:25.411307 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.411290    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 19:30:25.411662 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.411643    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 19:30:25.411706 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.411650    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6t64b\""
Jul 30 19:30:25.412271 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.412255    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 19:30:25.421960 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.421938    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"]
Jul 30 19:30:25.443668 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.443641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-tls-certs\") pod \"kube-auth-proxy-5b4b6cc975-kp2nh\" (UID: \"05ea1da3-9e19-42a4-aa1f-a6de6f74c027\") " pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:25.443760 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.443677    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9gn5\" (UniqueName: \"kubernetes.io/projected/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-kube-api-access-p9gn5\") pod \"kube-auth-proxy-5b4b6cc975-kp2nh\" (UID: \"05ea1da3-9e19-42a4-aa1f-a6de6f74c027\") " pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:25.443760 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.443714    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-tmp\") pod \"kube-auth-proxy-5b4b6cc975-kp2nh\" (UID: \"05ea1da3-9e19-42a4-aa1f-a6de6f74c027\") " pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:25.544515 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.544493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-tls-certs\") pod \"kube-auth-proxy-5b4b6cc975-kp2nh\" (UID: \"05ea1da3-9e19-42a4-aa1f-a6de6f74c027\") " pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:25.544600 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.544524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9gn5\" (UniqueName: \"kubernetes.io/projected/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-kube-api-access-p9gn5\") pod \"kube-auth-proxy-5b4b6cc975-kp2nh\" (UID: \"05ea1da3-9e19-42a4-aa1f-a6de6f74c027\") " pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:25.544600 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.544550    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-tmp\") pod \"kube-auth-proxy-5b4b6cc975-kp2nh\" (UID: \"05ea1da3-9e19-42a4-aa1f-a6de6f74c027\") " pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:25.544680 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:30:25.544630    2577 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 30 19:30:25.544718 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:30:25.544692    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-tls-certs podName:05ea1da3-9e19-42a4-aa1f-a6de6f74c027 nodeName:}" failed. No retries permitted until 2026-07-30 19:30:26.04467297 +0000 UTC m=+345.805428211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-tls-certs") pod "kube-auth-proxy-5b4b6cc975-kp2nh" (UID: "05ea1da3-9e19-42a4-aa1f-a6de6f74c027") : secret "kube-auth-proxy-tls" not found
Jul 30 19:30:25.546725 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.546702    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-tmp\") pod \"kube-auth-proxy-5b4b6cc975-kp2nh\" (UID: \"05ea1da3-9e19-42a4-aa1f-a6de6f74c027\") " pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:25.558432 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:25.558412    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9gn5\" (UniqueName: \"kubernetes.io/projected/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-kube-api-access-p9gn5\") pod \"kube-auth-proxy-5b4b6cc975-kp2nh\" (UID: \"05ea1da3-9e19-42a4-aa1f-a6de6f74c027\") " pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:26.047887 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:26.047861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-tls-certs\") pod \"kube-auth-proxy-5b4b6cc975-kp2nh\" (UID: \"05ea1da3-9e19-42a4-aa1f-a6de6f74c027\") " pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:26.050142 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:26.050115    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05ea1da3-9e19-42a4-aa1f-a6de6f74c027-tls-certs\") pod \"kube-auth-proxy-5b4b6cc975-kp2nh\" (UID: \"05ea1da3-9e19-42a4-aa1f-a6de6f74c027\") " pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:26.316215 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:26.316190    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"
Jul 30 19:30:26.440301 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:26.440276    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh"]
Jul 30 19:30:26.442879 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:30:26.442854    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ea1da3_9e19_42a4_aa1f_a6de6f74c027.slice/crio-396c7235585415bf1194b22f8883423261fee3fe6f548b86556c7c4fc7fd5fb9 WatchSource:0}: Error finding container 396c7235585415bf1194b22f8883423261fee3fe6f548b86556c7c4fc7fd5fb9: Status 404 returned error can't find the container with id 396c7235585415bf1194b22f8883423261fee3fe6f548b86556c7c4fc7fd5fb9
Jul 30 19:30:26.948540 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:26.948504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh" event={"ID":"05ea1da3-9e19-42a4-aa1f-a6de6f74c027","Type":"ContainerStarted","Data":"396c7235585415bf1194b22f8883423261fee3fe6f548b86556c7c4fc7fd5fb9"}
Jul 30 19:30:29.962161 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:29.962129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh" event={"ID":"05ea1da3-9e19-42a4-aa1f-a6de6f74c027","Type":"ContainerStarted","Data":"a7e474945c7285d033d937136700604f69b0c3800fddb5eda789836584047ed4"}
Jul 30 19:30:29.978994 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:30:29.978948    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b4b6cc975-kp2nh" podStartSLOduration=2.168318621 podStartE2EDuration="4.978933606s" totalImagesPullingTime="2.810614985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:30:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:30:29.977570133 +0000 UTC m=+349.738325395" watchObservedRunningTime="2026-07-30 19:30:29.978933606 +0000 UTC m=+349.739688864"
Jul 30 19:33:12.479107 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.479059    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:33:12.481104 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.481065    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt"
Jul 30 19:33:12.483362 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.483339    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 19:33:12.483468 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.483361    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 19:33:12.483468 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.483362    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 19:33:12.484178 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.484163    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gnqwl\""
Jul 30 19:33:12.491714 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.491686    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:33:12.507208 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.507189    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:33:12.513051 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.513029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqd4v\" (UniqueName: \"kubernetes.io/projected/9d6b89b3-e211-4e72-8069-4c3931d99f21-kube-api-access-fqd4v\") pod \"limitador-limitador-54d6d988d5-gkllt\" (UID: \"9d6b89b3-e211-4e72-8069-4c3931d99f21\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt"
Jul 30 19:33:12.513157 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.513108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d6b89b3-e211-4e72-8069-4c3931d99f21-config-file\") pod \"limitador-limitador-54d6d988d5-gkllt\" (UID: \"9d6b89b3-e211-4e72-8069-4c3931d99f21\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt"
Jul 30 19:33:12.613334 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.613306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d6b89b3-e211-4e72-8069-4c3931d99f21-config-file\") pod \"limitador-limitador-54d6d988d5-gkllt\" (UID: \"9d6b89b3-e211-4e72-8069-4c3931d99f21\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt"
Jul 30 19:33:12.613438 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.613350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqd4v\" (UniqueName: \"kubernetes.io/projected/9d6b89b3-e211-4e72-8069-4c3931d99f21-kube-api-access-fqd4v\") pod \"limitador-limitador-54d6d988d5-gkllt\" (UID: \"9d6b89b3-e211-4e72-8069-4c3931d99f21\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt"
Jul 30 19:33:12.613904 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.613881    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d6b89b3-e211-4e72-8069-4c3931d99f21-config-file\") pod \"limitador-limitador-54d6d988d5-gkllt\" (UID: \"9d6b89b3-e211-4e72-8069-4c3931d99f21\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt"
Jul 30 19:33:12.621153 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.621131    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqd4v\" (UniqueName: \"kubernetes.io/projected/9d6b89b3-e211-4e72-8069-4c3931d99f21-kube-api-access-fqd4v\") pod \"limitador-limitador-54d6d988d5-gkllt\" (UID: \"9d6b89b3-e211-4e72-8069-4c3931d99f21\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt"
Jul 30 19:33:12.791091 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.791016    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt"
Jul 30 19:33:12.912304 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.912270    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:33:12.915126 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:33:12.915099    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6b89b3_e211_4e72_8069_4c3931d99f21.slice/crio-3e6a26e9eb2c9e9b2a46ed3906f73725984fae5b671a8f14a7bc67761b8dcc55 WatchSource:0}: Error finding container 3e6a26e9eb2c9e9b2a46ed3906f73725984fae5b671a8f14a7bc67761b8dcc55: Status 404 returned error can't find the container with id 3e6a26e9eb2c9e9b2a46ed3906f73725984fae5b671a8f14a7bc67761b8dcc55
Jul 30 19:33:12.950561 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.950531    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-8nqxw"]
Jul 30 19:33:12.953225 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.953209    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8nqxw"
Jul 30 19:33:12.955399 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.955383    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-849zw\""
Jul 30 19:33:12.960453 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:12.960431    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8nqxw"]
Jul 30 19:33:13.017751 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.017733    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdxpp\" (UniqueName: \"kubernetes.io/projected/1b9b6f1d-da4f-4fc5-a069-d720ea372192-kube-api-access-sdxpp\") pod \"authorino-58cd955675-8nqxw\" (UID: \"1b9b6f1d-da4f-4fc5-a069-d720ea372192\") " pod="kuadrant-system/authorino-58cd955675-8nqxw"
Jul 30 19:33:13.061696 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.061671    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-4zklr"]
Jul 30 19:33:13.063892 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.063876    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4zklr"
Jul 30 19:33:13.070447 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.070428    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4zklr"]
Jul 30 19:33:13.118302 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.118273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czpf7\" (UniqueName: \"kubernetes.io/projected/04916121-36e2-4862-aefa-e08a06ac5a12-kube-api-access-czpf7\") pod \"authorino-75d556bd77-4zklr\" (UID: \"04916121-36e2-4862-aefa-e08a06ac5a12\") " pod="kuadrant-system/authorino-75d556bd77-4zklr"
Jul 30 19:33:13.118412 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.118330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdxpp\" (UniqueName: \"kubernetes.io/projected/1b9b6f1d-da4f-4fc5-a069-d720ea372192-kube-api-access-sdxpp\") pod \"authorino-58cd955675-8nqxw\" (UID: \"1b9b6f1d-da4f-4fc5-a069-d720ea372192\") " pod="kuadrant-system/authorino-58cd955675-8nqxw"
Jul 30 19:33:13.125692 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.125668    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdxpp\" (UniqueName: \"kubernetes.io/projected/1b9b6f1d-da4f-4fc5-a069-d720ea372192-kube-api-access-sdxpp\") pod \"authorino-58cd955675-8nqxw\" (UID: \"1b9b6f1d-da4f-4fc5-a069-d720ea372192\") " pod="kuadrant-system/authorino-58cd955675-8nqxw"
Jul 30 19:33:13.219267 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.219244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czpf7\" (UniqueName: \"kubernetes.io/projected/04916121-36e2-4862-aefa-e08a06ac5a12-kube-api-access-czpf7\") pod \"authorino-75d556bd77-4zklr\" (UID: \"04916121-36e2-4862-aefa-e08a06ac5a12\") " pod="kuadrant-system/authorino-75d556bd77-4zklr"
Jul 30 19:33:13.229406 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.229386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czpf7\" (UniqueName: \"kubernetes.io/projected/04916121-36e2-4862-aefa-e08a06ac5a12-kube-api-access-czpf7\") pod \"authorino-75d556bd77-4zklr\" (UID: \"04916121-36e2-4862-aefa-e08a06ac5a12\") " pod="kuadrant-system/authorino-75d556bd77-4zklr"
Jul 30 19:33:13.262169 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.262149    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8nqxw"
Jul 30 19:33:13.372029 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.372002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4zklr"
Jul 30 19:33:13.381399 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.381375    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8nqxw"]
Jul 30 19:33:13.383965 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:33:13.383939    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b9b6f1d_da4f_4fc5_a069_d720ea372192.slice/crio-6852d24df22192ec4bcc75669155274d93ff46a693b1de9ea9c336285778eea1 WatchSource:0}: Error finding container 6852d24df22192ec4bcc75669155274d93ff46a693b1de9ea9c336285778eea1: Status 404 returned error can't find the container with id 6852d24df22192ec4bcc75669155274d93ff46a693b1de9ea9c336285778eea1
Jul 30 19:33:13.497364 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.497253    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4zklr"]
Jul 30 19:33:13.500162 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:33:13.500132    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04916121_36e2_4862_aefa_e08a06ac5a12.slice/crio-e1bcf8ca64bb986fa12298c0416244783005611a51faf53644532647f34474df WatchSource:0}: Error finding container e1bcf8ca64bb986fa12298c0416244783005611a51faf53644532647f34474df: Status 404 returned error can't find the container with id e1bcf8ca64bb986fa12298c0416244783005611a51faf53644532647f34474df
Jul 30 19:33:13.517640 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.517607    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8nqxw" event={"ID":"1b9b6f1d-da4f-4fc5-a069-d720ea372192","Type":"ContainerStarted","Data":"6852d24df22192ec4bcc75669155274d93ff46a693b1de9ea9c336285778eea1"}
Jul 30 19:33:13.519085 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.519044    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4zklr" event={"ID":"04916121-36e2-4862-aefa-e08a06ac5a12","Type":"ContainerStarted","Data":"e1bcf8ca64bb986fa12298c0416244783005611a51faf53644532647f34474df"}
Jul 30 19:33:13.520604 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:13.520577    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt" event={"ID":"9d6b89b3-e211-4e72-8069-4c3931d99f21","Type":"ContainerStarted","Data":"3e6a26e9eb2c9e9b2a46ed3906f73725984fae5b671a8f14a7bc67761b8dcc55"}
Jul 30 19:33:18.550530 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:18.550489    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt" event={"ID":"9d6b89b3-e211-4e72-8069-4c3931d99f21","Type":"ContainerStarted","Data":"63c5184754e6c714b73c5c8e0d5a76805bd34f9312708e16f779fbeaac198dcd"}
Jul 30 19:33:18.550918 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:18.550669    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt"
Jul 30 19:33:18.551983 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:18.551958    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8nqxw" event={"ID":"1b9b6f1d-da4f-4fc5-a069-d720ea372192","Type":"ContainerStarted","Data":"84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586"}
Jul 30 19:33:18.553370 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:18.553344    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4zklr" event={"ID":"04916121-36e2-4862-aefa-e08a06ac5a12","Type":"ContainerStarted","Data":"0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af"}
Jul 30 19:33:18.567334 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:18.567295    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt" podStartSLOduration=1.707489778 podStartE2EDuration="6.567283356s" totalImagesPullingTime="4.859793578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:33:18.565597282 +0000 UTC m=+518.326352541" watchObservedRunningTime="2026-07-30 19:33:18.567283356 +0000 UTC m=+518.328038611"
Jul 30 19:33:18.578797 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:18.578763    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-4zklr" podStartSLOduration=1.309851326 podStartE2EDuration="5.578754349s" totalImagesPullingTime="4.268903023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:33:18.578530807 +0000 UTC m=+518.339286067" watchObservedRunningTime="2026-07-30 19:33:18.578754349 +0000 UTC m=+518.339509585"
Jul 30 19:33:18.592945 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:18.592902    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-8nqxw" podStartSLOduration=2.183895585 podStartE2EDuration="6.592887992s" totalImagesPullingTime="4.408992407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:33:18.592424185 +0000 UTC m=+518.353179456" watchObservedRunningTime="2026-07-30 19:33:18.592887992 +0000 UTC m=+518.353643258"
Jul 30 19:33:18.614498 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:18.614468    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8nqxw"]
Jul 30 19:33:20.560463 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:20.560424    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-8nqxw" podUID="1b9b6f1d-da4f-4fc5-a069-d720ea372192" containerName="authorino" containerID="cri-o://84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586" gracePeriod=30
Jul 30 19:33:20.796049 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:20.796026    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8nqxw"
Jul 30 19:33:20.877252 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:20.877195    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdxpp\" (UniqueName: \"kubernetes.io/projected/1b9b6f1d-da4f-4fc5-a069-d720ea372192-kube-api-access-sdxpp\") pod \"1b9b6f1d-da4f-4fc5-a069-d720ea372192\" (UID: \"1b9b6f1d-da4f-4fc5-a069-d720ea372192\") "
Jul 30 19:33:20.879248 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:20.879228    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b9b6f1d-da4f-4fc5-a069-d720ea372192-kube-api-access-sdxpp" (OuterVolumeSpecName: "kube-api-access-sdxpp") pod "1b9b6f1d-da4f-4fc5-a069-d720ea372192" (UID: "1b9b6f1d-da4f-4fc5-a069-d720ea372192"). InnerVolumeSpecName "kube-api-access-sdxpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:33:20.978268 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:20.978245    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdxpp\" (UniqueName: \"kubernetes.io/projected/1b9b6f1d-da4f-4fc5-a069-d720ea372192-kube-api-access-sdxpp\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:33:21.566006 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:21.565972    2577 generic.go:356] "Generic (PLEG): container finished" podID="1b9b6f1d-da4f-4fc5-a069-d720ea372192" containerID="84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586" exitCode=0
Jul 30 19:33:21.566418 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:21.566018    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8nqxw"
Jul 30 19:33:21.566418 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:21.566059    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8nqxw" event={"ID":"1b9b6f1d-da4f-4fc5-a069-d720ea372192","Type":"ContainerDied","Data":"84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586"}
Jul 30 19:33:21.566418 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:21.566125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8nqxw" event={"ID":"1b9b6f1d-da4f-4fc5-a069-d720ea372192","Type":"ContainerDied","Data":"6852d24df22192ec4bcc75669155274d93ff46a693b1de9ea9c336285778eea1"}
Jul 30 19:33:21.566838 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:21.566823    2577 scope.go:117] "RemoveContainer" containerID="84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586"
Jul 30 19:33:21.575296 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:21.575279    2577 scope.go:117] "RemoveContainer" containerID="84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586"
Jul 30 19:33:21.575541 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:33:21.575521    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586\": container with ID starting with 84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586 not found: ID does not exist" containerID="84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586"
Jul 30 19:33:21.575606 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:21.575553    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586"} err="failed to get container status \"84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586\": rpc error: code = NotFound desc = could not find container \"84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586\": container with ID starting with 84cb96212456256f6c15a3fcc5d3dff295e43055a7043d4e674691168e2d5586 not found: ID does not exist"
Jul 30 19:33:21.585256 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:21.585232    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8nqxw"]
Jul 30 19:33:21.587599 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:21.587579    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-8nqxw"]
Jul 30 19:33:22.782795 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:22.782764    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b9b6f1d-da4f-4fc5-a069-d720ea372192" path="/var/lib/kubelet/pods/1b9b6f1d-da4f-4fc5-a069-d720ea372192/volumes"
Jul 30 19:33:29.558462 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:29.558435    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-gkllt"
Jul 30 19:33:46.959484 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:46.959451    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pfwt2"]
Jul 30 19:33:46.960163 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:46.959729    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9b6f1d-da4f-4fc5-a069-d720ea372192" containerName="authorino"
Jul 30 19:33:46.960163 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:46.959748    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9b6f1d-da4f-4fc5-a069-d720ea372192" containerName="authorino"
Jul 30 19:33:46.960163 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:46.959811    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9b6f1d-da4f-4fc5-a069-d720ea372192" containerName="authorino"
Jul 30 19:33:46.962373 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:46.962352    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pfwt2"
Jul 30 19:33:46.980232 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:46.980207    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pfwt2"]
Jul 30 19:33:47.046457 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.046428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-226jf\" (UniqueName: \"kubernetes.io/projected/1848c4e9-28a7-4fb0-abc3-24d4176c8e1f-kube-api-access-226jf\") pod \"authorino-5d4b67c446-pfwt2\" (UID: \"1848c4e9-28a7-4fb0-abc3-24d4176c8e1f\") " pod="kuadrant-system/authorino-5d4b67c446-pfwt2"
Jul 30 19:33:47.146863 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.146836    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-226jf\" (UniqueName: \"kubernetes.io/projected/1848c4e9-28a7-4fb0-abc3-24d4176c8e1f-kube-api-access-226jf\") pod \"authorino-5d4b67c446-pfwt2\" (UID: \"1848c4e9-28a7-4fb0-abc3-24d4176c8e1f\") " pod="kuadrant-system/authorino-5d4b67c446-pfwt2"
Jul 30 19:33:47.154645 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.154619    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-226jf\" (UniqueName: \"kubernetes.io/projected/1848c4e9-28a7-4fb0-abc3-24d4176c8e1f-kube-api-access-226jf\") pod \"authorino-5d4b67c446-pfwt2\" (UID: \"1848c4e9-28a7-4fb0-abc3-24d4176c8e1f\") " pod="kuadrant-system/authorino-5d4b67c446-pfwt2"
Jul 30 19:33:47.204851 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.204824    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pfwt2"]
Jul 30 19:33:47.204994 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.204982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pfwt2"
Jul 30 19:33:47.232820 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.232798    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7899896548-zbd8h"]
Jul 30 19:33:47.235082 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.235053    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7899896548-zbd8h"
Jul 30 19:33:47.258416 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.258348    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7899896548-zbd8h"]
Jul 30 19:33:47.333410 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.333377    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pfwt2"]
Jul 30 19:33:47.336326 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:33:47.336299    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1848c4e9_28a7_4fb0_abc3_24d4176c8e1f.slice/crio-c0abcc89d511c531b5aed9557a7938a5b8e6728422657c85b9b80fec07ad9325 WatchSource:0}: Error finding container c0abcc89d511c531b5aed9557a7938a5b8e6728422657c85b9b80fec07ad9325: Status 404 returned error can't find the container with id c0abcc89d511c531b5aed9557a7938a5b8e6728422657c85b9b80fec07ad9325
Jul 30 19:33:47.348346 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.348323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gvz5\" (UniqueName: \"kubernetes.io/projected/296ab2a7-d1cf-4943-a7d1-c50549006b4d-kube-api-access-2gvz5\") pod \"authorino-7899896548-zbd8h\" (UID: \"296ab2a7-d1cf-4943-a7d1-c50549006b4d\") " pod="kuadrant-system/authorino-7899896548-zbd8h"
Jul 30 19:33:47.448763 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.448741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvz5\" (UniqueName: \"kubernetes.io/projected/296ab2a7-d1cf-4943-a7d1-c50549006b4d-kube-api-access-2gvz5\") pod \"authorino-7899896548-zbd8h\" (UID: \"296ab2a7-d1cf-4943-a7d1-c50549006b4d\") " pod="kuadrant-system/authorino-7899896548-zbd8h"
Jul 30 19:33:47.460248 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.460221    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gvz5\" (UniqueName: \"kubernetes.io/projected/296ab2a7-d1cf-4943-a7d1-c50549006b4d-kube-api-access-2gvz5\") pod \"authorino-7899896548-zbd8h\" (UID: \"296ab2a7-d1cf-4943-a7d1-c50549006b4d\") " pod="kuadrant-system/authorino-7899896548-zbd8h"
Jul 30 19:33:47.479533 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.479482    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7899896548-zbd8h"]
Jul 30 19:33:47.479646 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.479635    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7899896548-zbd8h"
Jul 30 19:33:47.510555 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.510528    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-556b6d5b44-vf4hz"]
Jul 30 19:33:47.513283 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.513264    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-556b6d5b44-vf4hz"
Jul 30 19:33:47.515426 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.515406    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 30 19:33:47.522427 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.522404    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-556b6d5b44-vf4hz"]
Jul 30 19:33:47.549367 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.549343    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv9tp\" (UniqueName: \"kubernetes.io/projected/e030bd94-1869-4401-adb8-c0e97ecb1745-kube-api-access-wv9tp\") pod \"authorino-556b6d5b44-vf4hz\" (UID: \"e030bd94-1869-4401-adb8-c0e97ecb1745\") " pod="kuadrant-system/authorino-556b6d5b44-vf4hz"
Jul 30 19:33:47.549479 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.549374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e030bd94-1869-4401-adb8-c0e97ecb1745-tls-cert\") pod \"authorino-556b6d5b44-vf4hz\" (UID: \"e030bd94-1869-4401-adb8-c0e97ecb1745\") " pod="kuadrant-system/authorino-556b6d5b44-vf4hz"
Jul 30 19:33:47.606948 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.606912    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7899896548-zbd8h"]
Jul 30 19:33:47.612673 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:33:47.612647    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod296ab2a7_d1cf_4943_a7d1_c50549006b4d.slice/crio-3f9d2c6e52628ac2b748832595e2bdbd9f21d527ff010b80af03f080c290bdb2 WatchSource:0}: Error finding container 3f9d2c6e52628ac2b748832595e2bdbd9f21d527ff010b80af03f080c290bdb2: Status 404 returned error can't find the container with id 3f9d2c6e52628ac2b748832595e2bdbd9f21d527ff010b80af03f080c290bdb2
Jul 30 19:33:47.649821 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.649799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv9tp\" (UniqueName: \"kubernetes.io/projected/e030bd94-1869-4401-adb8-c0e97ecb1745-kube-api-access-wv9tp\") pod \"authorino-556b6d5b44-vf4hz\" (UID: \"e030bd94-1869-4401-adb8-c0e97ecb1745\") " pod="kuadrant-system/authorino-556b6d5b44-vf4hz"
Jul 30 19:33:47.649920 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.649829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e030bd94-1869-4401-adb8-c0e97ecb1745-tls-cert\") pod \"authorino-556b6d5b44-vf4hz\" (UID: \"e030bd94-1869-4401-adb8-c0e97ecb1745\") " pod="kuadrant-system/authorino-556b6d5b44-vf4hz"
Jul 30 19:33:47.652054 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.652036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e030bd94-1869-4401-adb8-c0e97ecb1745-tls-cert\") pod \"authorino-556b6d5b44-vf4hz\" (UID: \"e030bd94-1869-4401-adb8-c0e97ecb1745\") " pod="kuadrant-system/authorino-556b6d5b44-vf4hz"
Jul 30 19:33:47.658318 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.658295    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv9tp\" (UniqueName: \"kubernetes.io/projected/e030bd94-1869-4401-adb8-c0e97ecb1745-kube-api-access-wv9tp\") pod \"authorino-556b6d5b44-vf4hz\" (UID: \"e030bd94-1869-4401-adb8-c0e97ecb1745\") " pod="kuadrant-system/authorino-556b6d5b44-vf4hz"
Jul 30 19:33:47.666242 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.666217    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pfwt2" event={"ID":"1848c4e9-28a7-4fb0-abc3-24d4176c8e1f","Type":"ContainerStarted","Data":"c0abcc89d511c531b5aed9557a7938a5b8e6728422657c85b9b80fec07ad9325"}
Jul 30 19:33:47.667368 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.667347    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7899896548-zbd8h" event={"ID":"296ab2a7-d1cf-4943-a7d1-c50549006b4d","Type":"ContainerStarted","Data":"3f9d2c6e52628ac2b748832595e2bdbd9f21d527ff010b80af03f080c290bdb2"}
Jul 30 19:33:47.824629 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.824605    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-556b6d5b44-vf4hz"
Jul 30 19:33:47.951438 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:47.951411    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-556b6d5b44-vf4hz"]
Jul 30 19:33:47.953774 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:33:47.953743    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode030bd94_1869_4401_adb8_c0e97ecb1745.slice/crio-240d075081f7dd394e5bdcfca7e0001392cc16e3c90a3a6927255482b9780048 WatchSource:0}: Error finding container 240d075081f7dd394e5bdcfca7e0001392cc16e3c90a3a6927255482b9780048: Status 404 returned error can't find the container with id 240d075081f7dd394e5bdcfca7e0001392cc16e3c90a3a6927255482b9780048
Jul 30 19:33:48.672985 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:48.672886    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7899896548-zbd8h" event={"ID":"296ab2a7-d1cf-4943-a7d1-c50549006b4d","Type":"ContainerStarted","Data":"3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6"}
Jul 30 19:33:48.672985 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:48.672892    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7899896548-zbd8h" podUID="296ab2a7-d1cf-4943-a7d1-c50549006b4d" containerName="authorino" containerID="cri-o://3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6" gracePeriod=30
Jul 30 19:33:48.674544 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:48.674514    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-556b6d5b44-vf4hz" event={"ID":"e030bd94-1869-4401-adb8-c0e97ecb1745","Type":"ContainerStarted","Data":"240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b"}
Jul 30 19:33:48.674675 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:48.674553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-556b6d5b44-vf4hz" event={"ID":"e030bd94-1869-4401-adb8-c0e97ecb1745","Type":"ContainerStarted","Data":"240d075081f7dd394e5bdcfca7e0001392cc16e3c90a3a6927255482b9780048"}
Jul 30 19:33:48.676111 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:48.676085    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pfwt2" event={"ID":"1848c4e9-28a7-4fb0-abc3-24d4176c8e1f","Type":"ContainerStarted","Data":"5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492"}
Jul 30 19:33:48.676246 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:48.676191    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-pfwt2" podUID="1848c4e9-28a7-4fb0-abc3-24d4176c8e1f" containerName="authorino" containerID="cri-o://5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492" gracePeriod=30
Jul 30 19:33:48.688983 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:48.688914    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7899896548-zbd8h" podStartSLOduration=1.259446074 podStartE2EDuration="1.688898753s" totalImagesPullingTime="429.452679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:33:48.688221835 +0000 UTC m=+548.448977093" watchObservedRunningTime="2026-07-30 19:33:48.688898753 +0000 UTC m=+548.449654043"
Jul 30 19:33:48.704347 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:48.704302    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-pfwt2" podStartSLOduration=2.359808669 podStartE2EDuration="2.704285242s" totalImagesPullingTime="344.476573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:33:48.703984706 +0000 UTC m=+548.464739969" watchObservedRunningTime="2026-07-30 19:33:48.704285242 +0000 UTC m=+548.465040500"
Jul 30 19:33:48.720856 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:48.720819    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-556b6d5b44-vf4hz" podStartSLOduration=1.390295653 podStartE2EDuration="1.720806995s" totalImagesPullingTime="330.511342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:33:48.719696785 +0000 UTC m=+548.480452044" watchObservedRunningTime="2026-07-30 19:33:48.720806995 +0000 UTC m=+548.481562253"
Jul 30 19:33:48.745200 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:48.745173    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4zklr"]
Jul 30 19:33:48.745389 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:48.745365    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-4zklr" podUID="04916121-36e2-4862-aefa-e08a06ac5a12" containerName="authorino" containerID="cri-o://0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af" gracePeriod=30
Jul 30 19:33:49.000031 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.000004    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pfwt2"
Jul 30 19:33:49.023382 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.023360    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7899896548-zbd8h"
Jul 30 19:33:49.031931 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.031901    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4zklr"
Jul 30 19:33:49.062249 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.062222    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gvz5\" (UniqueName: \"kubernetes.io/projected/296ab2a7-d1cf-4943-a7d1-c50549006b4d-kube-api-access-2gvz5\") pod \"296ab2a7-d1cf-4943-a7d1-c50549006b4d\" (UID: \"296ab2a7-d1cf-4943-a7d1-c50549006b4d\") "
Jul 30 19:33:49.062364 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.062308    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-226jf\" (UniqueName: \"kubernetes.io/projected/1848c4e9-28a7-4fb0-abc3-24d4176c8e1f-kube-api-access-226jf\") pod \"1848c4e9-28a7-4fb0-abc3-24d4176c8e1f\" (UID: \"1848c4e9-28a7-4fb0-abc3-24d4176c8e1f\") "
Jul 30 19:33:49.062364 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.062355    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czpf7\" (UniqueName: \"kubernetes.io/projected/04916121-36e2-4862-aefa-e08a06ac5a12-kube-api-access-czpf7\") pod \"04916121-36e2-4862-aefa-e08a06ac5a12\" (UID: \"04916121-36e2-4862-aefa-e08a06ac5a12\") "
Jul 30 19:33:49.064848 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.064820    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1848c4e9-28a7-4fb0-abc3-24d4176c8e1f-kube-api-access-226jf" (OuterVolumeSpecName: "kube-api-access-226jf") pod "1848c4e9-28a7-4fb0-abc3-24d4176c8e1f" (UID: "1848c4e9-28a7-4fb0-abc3-24d4176c8e1f"). InnerVolumeSpecName "kube-api-access-226jf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:33:49.064925 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.064851    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/296ab2a7-d1cf-4943-a7d1-c50549006b4d-kube-api-access-2gvz5" (OuterVolumeSpecName: "kube-api-access-2gvz5") pod "296ab2a7-d1cf-4943-a7d1-c50549006b4d" (UID: "296ab2a7-d1cf-4943-a7d1-c50549006b4d"). InnerVolumeSpecName "kube-api-access-2gvz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:33:49.065045 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.065025    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04916121-36e2-4862-aefa-e08a06ac5a12-kube-api-access-czpf7" (OuterVolumeSpecName: "kube-api-access-czpf7") pod "04916121-36e2-4862-aefa-e08a06ac5a12" (UID: "04916121-36e2-4862-aefa-e08a06ac5a12"). InnerVolumeSpecName "kube-api-access-czpf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:33:49.163802 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.163779    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-226jf\" (UniqueName: \"kubernetes.io/projected/1848c4e9-28a7-4fb0-abc3-24d4176c8e1f-kube-api-access-226jf\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:33:49.163892 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.163805    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czpf7\" (UniqueName: \"kubernetes.io/projected/04916121-36e2-4862-aefa-e08a06ac5a12-kube-api-access-czpf7\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:33:49.163892 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.163820    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gvz5\" (UniqueName: \"kubernetes.io/projected/296ab2a7-d1cf-4943-a7d1-c50549006b4d-kube-api-access-2gvz5\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:33:49.681771 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.681735    2577 generic.go:356] "Generic (PLEG): container finished" podID="1848c4e9-28a7-4fb0-abc3-24d4176c8e1f" containerID="5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492" exitCode=0
Jul 30 19:33:49.682239 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.681773    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pfwt2"
Jul 30 19:33:49.682239 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.681820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pfwt2" event={"ID":"1848c4e9-28a7-4fb0-abc3-24d4176c8e1f","Type":"ContainerDied","Data":"5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492"}
Jul 30 19:33:49.682239 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.681866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pfwt2" event={"ID":"1848c4e9-28a7-4fb0-abc3-24d4176c8e1f","Type":"ContainerDied","Data":"c0abcc89d511c531b5aed9557a7938a5b8e6728422657c85b9b80fec07ad9325"}
Jul 30 19:33:49.682239 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.681882    2577 scope.go:117] "RemoveContainer" containerID="5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492"
Jul 30 19:33:49.683178 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.683022    2577 generic.go:356] "Generic (PLEG): container finished" podID="04916121-36e2-4862-aefa-e08a06ac5a12" containerID="0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af" exitCode=0
Jul 30 19:33:49.683178 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.683093    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4zklr"
Jul 30 19:33:49.683178 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.683110    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4zklr" event={"ID":"04916121-36e2-4862-aefa-e08a06ac5a12","Type":"ContainerDied","Data":"0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af"}
Jul 30 19:33:49.683178 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.683144    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4zklr" event={"ID":"04916121-36e2-4862-aefa-e08a06ac5a12","Type":"ContainerDied","Data":"e1bcf8ca64bb986fa12298c0416244783005611a51faf53644532647f34474df"}
Jul 30 19:33:49.684470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.684448    2577 generic.go:356] "Generic (PLEG): container finished" podID="296ab2a7-d1cf-4943-a7d1-c50549006b4d" containerID="3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6" exitCode=0
Jul 30 19:33:49.684558 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.684487    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7899896548-zbd8h"
Jul 30 19:33:49.684558 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.684503    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7899896548-zbd8h" event={"ID":"296ab2a7-d1cf-4943-a7d1-c50549006b4d","Type":"ContainerDied","Data":"3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6"}
Jul 30 19:33:49.684558 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.684537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7899896548-zbd8h" event={"ID":"296ab2a7-d1cf-4943-a7d1-c50549006b4d","Type":"ContainerDied","Data":"3f9d2c6e52628ac2b748832595e2bdbd9f21d527ff010b80af03f080c290bdb2"}
Jul 30 19:33:49.692088 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.692056    2577 scope.go:117] "RemoveContainer" containerID="5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492"
Jul 30 19:33:49.692453 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:33:49.692430    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492\": container with ID starting with 5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492 not found: ID does not exist" containerID="5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492"
Jul 30 19:33:49.692523 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.692464    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492"} err="failed to get container status \"5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492\": rpc error: code = NotFound desc = could not find container \"5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492\": container with ID starting with 5f505952cd38d02fc4f8a68b72c7d2012a835f6da66c491e08d120c71536b492 not found: ID does not exist"
Jul 30 19:33:49.692523 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.692488    2577 scope.go:117] "RemoveContainer" containerID="0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af"
Jul 30 19:33:49.700681 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.700667    2577 scope.go:117] "RemoveContainer" containerID="0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af"
Jul 30 19:33:49.700933 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:33:49.700913    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af\": container with ID starting with 0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af not found: ID does not exist" containerID="0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af"
Jul 30 19:33:49.700984 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.700940    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af"} err="failed to get container status \"0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af\": rpc error: code = NotFound desc = could not find container \"0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af\": container with ID starting with 0d649ebb2a91298665568e5b27dbe82a8c9f1642d2acb11e7db833c2fcf010af not found: ID does not exist"
Jul 30 19:33:49.700984 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.700958    2577 scope.go:117] "RemoveContainer" containerID="3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6"
Jul 30 19:33:49.707489 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.707472    2577 scope.go:117] "RemoveContainer" containerID="3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6"
Jul 30 19:33:49.707710 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:33:49.707691    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6\": container with ID starting with 3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6 not found: ID does not exist" containerID="3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6"
Jul 30 19:33:49.707780 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.707718    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6"} err="failed to get container status \"3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6\": rpc error: code = NotFound desc = could not find container \"3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6\": container with ID starting with 3257ff996d1d6817bdac14b2742e022a25a1a0e0d2fcebac93718d72b88b57f6 not found: ID does not exist"
Jul 30 19:33:49.709595 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.709575    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pfwt2"]
Jul 30 19:33:49.712768 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.712750    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pfwt2"]
Jul 30 19:33:49.722698 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.722679    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7899896548-zbd8h"]
Jul 30 19:33:49.726173 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.726153    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7899896548-zbd8h"]
Jul 30 19:33:49.735291 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.735272    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4zklr"]
Jul 30 19:33:49.740521 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:49.740501    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4zklr"]
Jul 30 19:33:50.783148 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:50.783117    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04916121-36e2-4862-aefa-e08a06ac5a12" path="/var/lib/kubelet/pods/04916121-36e2-4862-aefa-e08a06ac5a12/volumes"
Jul 30 19:33:50.783545 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:50.783513    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1848c4e9-28a7-4fb0-abc3-24d4176c8e1f" path="/var/lib/kubelet/pods/1848c4e9-28a7-4fb0-abc3-24d4176c8e1f/volumes"
Jul 30 19:33:50.783859 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:33:50.783844    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="296ab2a7-d1cf-4943-a7d1-c50549006b4d" path="/var/lib/kubelet/pods/296ab2a7-d1cf-4943-a7d1-c50549006b4d/volumes"
Jul 30 19:34:30.473308 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:34:30.473276    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:34:52.766458 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:34:52.763893    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:35:07.166532 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:07.166458    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:35:19.367103 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:19.367053    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:35:27.167741 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:27.167703    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:35:37.560780 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:37.560747    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:35:39.744943 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.744908    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"]
Jul 30 19:35:39.745336 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.745198    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="296ab2a7-d1cf-4943-a7d1-c50549006b4d" containerName="authorino"
Jul 30 19:35:39.745336 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.745210    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="296ab2a7-d1cf-4943-a7d1-c50549006b4d" containerName="authorino"
Jul 30 19:35:39.745336 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.745230    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1848c4e9-28a7-4fb0-abc3-24d4176c8e1f" containerName="authorino"
Jul 30 19:35:39.745336 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.745235    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1848c4e9-28a7-4fb0-abc3-24d4176c8e1f" containerName="authorino"
Jul 30 19:35:39.745336 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.745243    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04916121-36e2-4862-aefa-e08a06ac5a12" containerName="authorino"
Jul 30 19:35:39.745336 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.745248    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="04916121-36e2-4862-aefa-e08a06ac5a12" containerName="authorino"
Jul 30 19:35:39.745336 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.745289    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1848c4e9-28a7-4fb0-abc3-24d4176c8e1f" containerName="authorino"
Jul 30 19:35:39.745336 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.745297    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="296ab2a7-d1cf-4943-a7d1-c50549006b4d" containerName="authorino"
Jul 30 19:35:39.745336 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.745305    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="04916121-36e2-4862-aefa-e08a06ac5a12" containerName="authorino"
Jul 30 19:35:39.747505 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.747484    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.749858 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.749836    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 19:35:39.750845 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.750823    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 30 19:35:39.750954 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.750857    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-78nfb\""
Jul 30 19:35:39.751199 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.751152    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 19:35:39.758114 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.758092    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"]
Jul 30 19:35:39.880441 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.880416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.880549 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.880448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drzmc\" (UniqueName: \"kubernetes.io/projected/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-kube-api-access-drzmc\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.880549 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.880510    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.880549 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.880546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.880678 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.880572    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.880678 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.880588    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.981649 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.981623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drzmc\" (UniqueName: \"kubernetes.io/projected/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-kube-api-access-drzmc\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.981752 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.981658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.981752 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.981694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.981752 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.981725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.981905 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.981751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.981905 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.981814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.982092 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.982053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.982161 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.982115    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.982199 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.982174    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.983749 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.983730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.984105 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.984088    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:39.990035 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:39.990007    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drzmc\" (UniqueName: \"kubernetes.io/projected/98667ffa-8b3d-4f77-bdf1-82a6bf75177c-kube-api-access-drzmc\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jtws7\" (UID: \"98667ffa-8b3d-4f77-bdf1-82a6bf75177c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:40.057011 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:40.056987    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:40.183896 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:40.183871    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"]
Jul 30 19:35:40.186335 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:35:40.186306    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98667ffa_8b3d_4f77_bdf1_82a6bf75177c.slice/crio-7409a177b7eb498c6b652be4e43c8ac435d1811cf1ddc4802d92f3892a4ef3fa WatchSource:0}: Error finding container 7409a177b7eb498c6b652be4e43c8ac435d1811cf1ddc4802d92f3892a4ef3fa: Status 404 returned error can't find the container with id 7409a177b7eb498c6b652be4e43c8ac435d1811cf1ddc4802d92f3892a4ef3fa
Jul 30 19:35:40.187794 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:40.187772    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 19:35:41.088737 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:41.088697    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7" event={"ID":"98667ffa-8b3d-4f77-bdf1-82a6bf75177c","Type":"ContainerStarted","Data":"7409a177b7eb498c6b652be4e43c8ac435d1811cf1ddc4802d92f3892a4ef3fa"}
Jul 30 19:35:42.093670 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:42.093637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7" event={"ID":"98667ffa-8b3d-4f77-bdf1-82a6bf75177c","Type":"ContainerStarted","Data":"d7631aae50489fe235b13f67be40426a9ec673b54d781a552b0235257411b08e"}
Jul 30 19:35:42.111458 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:42.111410    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7" podStartSLOduration=2.101934614 podStartE2EDuration="3.111396831s" totalImagesPullingTime="1.009462217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:35:42.109442235 +0000 UTC m=+661.870197493" watchObservedRunningTime="2026-07-30 19:35:42.111396831 +0000 UTC m=+661.872152088"
Jul 30 19:35:48.756664 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:48.756633    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:35:50.058079 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:50.058045    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:50.058465 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:50.058120    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:50.069998 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:50.069976    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:35:50.135727 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:35:50.135701    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jtws7"
Jul 30 19:36:58.450599 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:58.450562    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8d8b8fb6f-qtrd8"]
Jul 30 19:36:58.455104 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:58.454947    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8d8b8fb6f-qtrd8"
Jul 30 19:36:58.461466 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:58.461435    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8d8b8fb6f-qtrd8"]
Jul 30 19:36:58.590491 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:58.590462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9593661e-3eda-4fc5-a1f4-52da8cb9d919-tls-cert\") pod \"authorino-8d8b8fb6f-qtrd8\" (UID: \"9593661e-3eda-4fc5-a1f4-52da8cb9d919\") " pod="kuadrant-system/authorino-8d8b8fb6f-qtrd8"
Jul 30 19:36:58.590491 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:58.590492    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shwx8\" (UniqueName: \"kubernetes.io/projected/9593661e-3eda-4fc5-a1f4-52da8cb9d919-kube-api-access-shwx8\") pod \"authorino-8d8b8fb6f-qtrd8\" (UID: \"9593661e-3eda-4fc5-a1f4-52da8cb9d919\") " pod="kuadrant-system/authorino-8d8b8fb6f-qtrd8"
Jul 30 19:36:58.691729 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:58.691705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9593661e-3eda-4fc5-a1f4-52da8cb9d919-tls-cert\") pod \"authorino-8d8b8fb6f-qtrd8\" (UID: \"9593661e-3eda-4fc5-a1f4-52da8cb9d919\") " pod="kuadrant-system/authorino-8d8b8fb6f-qtrd8"
Jul 30 19:36:58.691815 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:58.691731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shwx8\" (UniqueName: \"kubernetes.io/projected/9593661e-3eda-4fc5-a1f4-52da8cb9d919-kube-api-access-shwx8\") pod \"authorino-8d8b8fb6f-qtrd8\" (UID: \"9593661e-3eda-4fc5-a1f4-52da8cb9d919\") " pod="kuadrant-system/authorino-8d8b8fb6f-qtrd8"
Jul 30 19:36:58.693966 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:58.693940    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9593661e-3eda-4fc5-a1f4-52da8cb9d919-tls-cert\") pod \"authorino-8d8b8fb6f-qtrd8\" (UID: \"9593661e-3eda-4fc5-a1f4-52da8cb9d919\") " pod="kuadrant-system/authorino-8d8b8fb6f-qtrd8"
Jul 30 19:36:58.698890 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:58.698863    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shwx8\" (UniqueName: \"kubernetes.io/projected/9593661e-3eda-4fc5-a1f4-52da8cb9d919-kube-api-access-shwx8\") pod \"authorino-8d8b8fb6f-qtrd8\" (UID: \"9593661e-3eda-4fc5-a1f4-52da8cb9d919\") " pod="kuadrant-system/authorino-8d8b8fb6f-qtrd8"
Jul 30 19:36:58.765736 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:58.765688    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8d8b8fb6f-qtrd8"
Jul 30 19:36:58.891615 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:58.891582    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8d8b8fb6f-qtrd8"]
Jul 30 19:36:58.894479 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:36:58.894454    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9593661e_3eda_4fc5_a1f4_52da8cb9d919.slice/crio-0707c425744ca08f07218a8840899eca2c98b395e7cb265fc0b80693c857d5e5 WatchSource:0}: Error finding container 0707c425744ca08f07218a8840899eca2c98b395e7cb265fc0b80693c857d5e5: Status 404 returned error can't find the container with id 0707c425744ca08f07218a8840899eca2c98b395e7cb265fc0b80693c857d5e5
Jul 30 19:36:59.383358 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:36:59.383330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8d8b8fb6f-qtrd8" event={"ID":"9593661e-3eda-4fc5-a1f4-52da8cb9d919","Type":"ContainerStarted","Data":"0707c425744ca08f07218a8840899eca2c98b395e7cb265fc0b80693c857d5e5"}
Jul 30 19:37:00.389903 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:00.389861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8d8b8fb6f-qtrd8" event={"ID":"9593661e-3eda-4fc5-a1f4-52da8cb9d919","Type":"ContainerStarted","Data":"92af87b73005e6e04fdf89362806dc3fd039ea1253afc5a5f48152ec850ab799"}
Jul 30 19:37:00.405300 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:00.405240    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8d8b8fb6f-qtrd8" podStartSLOduration=1.960944473 podStartE2EDuration="2.405219875s" totalImagesPullingTime="444.275402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:37:00.404825055 +0000 UTC m=+740.165580316" watchObservedRunningTime="2026-07-30 19:37:00.405219875 +0000 UTC m=+740.165975137"
Jul 30 19:37:00.433922 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:00.433886    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-556b6d5b44-vf4hz"]
Jul 30 19:37:00.434434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:00.434394    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-556b6d5b44-vf4hz" podUID="e030bd94-1869-4401-adb8-c0e97ecb1745" containerName="authorino" containerID="cri-o://240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b" gracePeriod=30
Jul 30 19:37:00.676133 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:00.676109    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-556b6d5b44-vf4hz"
Jul 30 19:37:00.807287 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:00.807253    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e030bd94-1869-4401-adb8-c0e97ecb1745-tls-cert\") pod \"e030bd94-1869-4401-adb8-c0e97ecb1745\" (UID: \"e030bd94-1869-4401-adb8-c0e97ecb1745\") "
Jul 30 19:37:00.807449 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:00.807310    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wv9tp\" (UniqueName: \"kubernetes.io/projected/e030bd94-1869-4401-adb8-c0e97ecb1745-kube-api-access-wv9tp\") pod \"e030bd94-1869-4401-adb8-c0e97ecb1745\" (UID: \"e030bd94-1869-4401-adb8-c0e97ecb1745\") "
Jul 30 19:37:00.809343 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:00.809308    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e030bd94-1869-4401-adb8-c0e97ecb1745-kube-api-access-wv9tp" (OuterVolumeSpecName: "kube-api-access-wv9tp") pod "e030bd94-1869-4401-adb8-c0e97ecb1745" (UID: "e030bd94-1869-4401-adb8-c0e97ecb1745"). InnerVolumeSpecName "kube-api-access-wv9tp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:37:00.816476 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:00.816446    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e030bd94-1869-4401-adb8-c0e97ecb1745-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e030bd94-1869-4401-adb8-c0e97ecb1745" (UID: "e030bd94-1869-4401-adb8-c0e97ecb1745"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:37:00.908639 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:00.908549    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e030bd94-1869-4401-adb8-c0e97ecb1745-tls-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:37:00.908639 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:00.908584    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wv9tp\" (UniqueName: \"kubernetes.io/projected/e030bd94-1869-4401-adb8-c0e97ecb1745-kube-api-access-wv9tp\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:37:01.394696 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:01.394662    2577 generic.go:356] "Generic (PLEG): container finished" podID="e030bd94-1869-4401-adb8-c0e97ecb1745" containerID="240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b" exitCode=0
Jul 30 19:37:01.395193 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:01.394712    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-556b6d5b44-vf4hz"
Jul 30 19:37:01.395193 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:01.394747    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-556b6d5b44-vf4hz" event={"ID":"e030bd94-1869-4401-adb8-c0e97ecb1745","Type":"ContainerDied","Data":"240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b"}
Jul 30 19:37:01.395193 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:01.394785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-556b6d5b44-vf4hz" event={"ID":"e030bd94-1869-4401-adb8-c0e97ecb1745","Type":"ContainerDied","Data":"240d075081f7dd394e5bdcfca7e0001392cc16e3c90a3a6927255482b9780048"}
Jul 30 19:37:01.395193 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:01.394817    2577 scope.go:117] "RemoveContainer" containerID="240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b"
Jul 30 19:37:01.403284 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:01.403257    2577 scope.go:117] "RemoveContainer" containerID="240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b"
Jul 30 19:37:01.403600 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:37:01.403568    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b\": container with ID starting with 240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b not found: ID does not exist" containerID="240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b"
Jul 30 19:37:01.403688 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:01.403613    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b"} err="failed to get container status \"240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b\": rpc error: code = NotFound desc = could not find container \"240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b\": container with ID starting with 240558583e6a29544b2359ddde0529ce977333a0a138cfca78669606ae7f7f3b not found: ID does not exist"
Jul 30 19:37:01.415166 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:01.415145    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-556b6d5b44-vf4hz"]
Jul 30 19:37:01.419192 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:01.419173    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-556b6d5b44-vf4hz"]
Jul 30 19:37:02.782765 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:02.782735    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e030bd94-1869-4401-adb8-c0e97ecb1745" path="/var/lib/kubelet/pods/e030bd94-1869-4401-adb8-c0e97ecb1745/volumes"
Jul 30 19:37:28.475711 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:28.475670    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:37:38.159859 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:38.159815    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:37:49.578525 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:37:49.578487    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:39:31.460054 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:39:31.460015    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:39:39.963585 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:39:39.963508    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:40:30.467254 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:40:30.467222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:40:40.176598 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:40:40.176562    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:40:55.563514 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:40:55.563480    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:41:04.859717 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:41:04.859632    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:41:15.355612 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:41:15.355576    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:41:25.243565 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:41:25.243526    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:41:35.757944 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:41:35.757907    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:41:45.859936 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:41:45.859904    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:42:07.967262 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:42:07.967230    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:42:18.863614 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:42:18.863582    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:42:41.266320 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:42:41.266242    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:42:51.763162 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:42:51.763125    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:43:08.869452 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:43:08.869418    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:43:28.769732 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:43:28.769700    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:43:43.869043 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:43:43.868999    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:43:53.471557 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:43:53.471513    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:44:02.763665 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:44:02.763587    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:44:12.265931 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:44:12.265894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:44:21.861706 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:44:21.861655    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:44:43.161363 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:44:43.161332    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:44:53.664393 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:44:53.664362    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:45:03.357088 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:45:03.357040    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:47:13.755815 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:47:13.755726    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:47:23.973915 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:47:23.973881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:47:45.069214 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:47:45.069180    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:48:35.562269 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:48:35.562194    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:48:47.659552 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:48:47.659518    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:48:55.870706 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:48:55.870670    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:49:06.763229 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:49:06.763199    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:49:17.470505 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:49:17.470474    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:49:26.665024 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:49:26.664988    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:49:37.862536 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:49:37.862507    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:49:49.764586 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:49:49.764550    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:49:58.766470 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:49:58.766435    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:50:14.565124 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:14.565047    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:50:22.565745 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.565709    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"]
Jul 30 19:50:22.566150 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.565952    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e030bd94-1869-4401-adb8-c0e97ecb1745" containerName="authorino"
Jul 30 19:50:22.566150 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.565963    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e030bd94-1869-4401-adb8-c0e97ecb1745" containerName="authorino"
Jul 30 19:50:22.566150 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.566020    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e030bd94-1869-4401-adb8-c0e97ecb1745" containerName="authorino"
Jul 30 19:50:22.568700 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.568681    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.570884 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.570863    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-3dbce65f-kserve-self-signed-certs\""
Jul 30 19:50:22.589522 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.589499    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"]
Jul 30 19:50:22.653692 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.653668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-tmp-dir\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.653795 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.653703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qng9g\" (UniqueName: \"kubernetes.io/projected/887ec1b5-c007-4109-9742-0ded5cdf86d0-kube-api-access-qng9g\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.653795 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.653726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/887ec1b5-c007-4109-9742-0ded5cdf86d0-tls-certs\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.653795 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.653741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-home\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.653911 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.653796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-model-cache\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.653911 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.653853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-dshm\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.754919 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.754899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qng9g\" (UniqueName: \"kubernetes.io/projected/887ec1b5-c007-4109-9742-0ded5cdf86d0-kube-api-access-qng9g\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.754998 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.754924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/887ec1b5-c007-4109-9742-0ded5cdf86d0-tls-certs\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.754998 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.754940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-home\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.754998 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.754960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-model-cache\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.755169 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.754997    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-dshm\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.755169 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.755031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-tmp-dir\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.755384 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.755361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-home\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.755449 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.755384    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-tmp-dir\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.755449 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.755418    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-model-cache\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.757234 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.757209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-dshm\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.757411 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.757393    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/887ec1b5-c007-4109-9742-0ded5cdf86d0-tls-certs\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.762407 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.762386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qng9g\" (UniqueName: \"kubernetes.io/projected/887ec1b5-c007-4109-9742-0ded5cdf86d0-kube-api-access-qng9g\") pod \"e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") " pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:22.877504 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:22.877448    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:23.000749 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:23.000676    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"]
Jul 30 19:50:23.002910 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:50:23.002874    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod887ec1b5_c007_4109_9742_0ded5cdf86d0.slice/crio-f3b3248c3e011bd8f344ae6930843165a165b9a470bf7f7b4d3779c6b3ffcee5 WatchSource:0}: Error finding container f3b3248c3e011bd8f344ae6930843165a165b9a470bf7f7b4d3779c6b3ffcee5: Status 404 returned error can't find the container with id f3b3248c3e011bd8f344ae6930843165a165b9a470bf7f7b4d3779c6b3ffcee5
Jul 30 19:50:23.367740 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:23.367705    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t" event={"ID":"887ec1b5-c007-4109-9742-0ded5cdf86d0","Type":"ContainerStarted","Data":"5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e"}
Jul 30 19:50:23.367893 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:23.367754    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t" event={"ID":"887ec1b5-c007-4109-9742-0ded5cdf86d0","Type":"ContainerStarted","Data":"f3b3248c3e011bd8f344ae6930843165a165b9a470bf7f7b4d3779c6b3ffcee5"}
Jul 30 19:50:24.769195 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:24.769142    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t" podStartSLOduration=2.769123259 podStartE2EDuration="2.769123259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:50:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:50:23.383498224 +0000 UTC m=+1543.144253483" watchObservedRunningTime="2026-07-30 19:50:24.769123259 +0000 UTC m=+1544.529878516"
Jul 30 19:50:24.769798 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:24.769781    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:50:32.877856 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:32.877782    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:32.877856 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:32.877825    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:32.890033 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:32.890014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:33.272365 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.272299    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"]
Jul 30 19:50:33.275976 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.275957    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.278355 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.278336    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-3dbce65f-kserve-self-signed-certs\""
Jul 30 19:50:33.283920 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.283897    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"]
Jul 30 19:50:33.419691 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.419663    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:50:33.423523 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.423505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tmp-dir\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.423581 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.423533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqv4k\" (UniqueName: \"kubernetes.io/projected/f169fe3a-4dc8-43bd-9017-dc5124b327aa-kube-api-access-rqv4k\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.423581 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.423553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-model-cache\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.423581 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.423577    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-dshm\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.423678 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.423643    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-home\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.423711 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.423686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tls-certs\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.524346 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.524269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-home\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.524346 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.524314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tls-certs\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.524559 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.524495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tmp-dir\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.524676 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.524648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqv4k\" (UniqueName: \"kubernetes.io/projected/f169fe3a-4dc8-43bd-9017-dc5124b327aa-kube-api-access-rqv4k\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.524744 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.524691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-home\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.524744 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.524697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-model-cache\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.524836 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.524772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-dshm\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.524988 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.524967    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-model-cache\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.525176 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.525157    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tmp-dir\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.526880 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.526855    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tls-certs\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.526968 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.526948    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-dshm\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.532766 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.532736    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqv4k\" (UniqueName: \"kubernetes.io/projected/f169fe3a-4dc8-43bd-9017-dc5124b327aa-kube-api-access-rqv4k\") pod \"e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") " pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.585929 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.585893    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:33.712818 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:33.712737    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"]
Jul 30 19:50:33.715520 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:50:33.715490    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf169fe3a_4dc8_43bd_9017_dc5124b327aa.slice/crio-2fb54e482cb83e587100438fb8680cae212f764c8c393a8e6943790dec312d5d WatchSource:0}: Error finding container 2fb54e482cb83e587100438fb8680cae212f764c8c393a8e6943790dec312d5d: Status 404 returned error can't find the container with id 2fb54e482cb83e587100438fb8680cae212f764c8c393a8e6943790dec312d5d
Jul 30 19:50:34.414126 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:34.414064    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c" event={"ID":"f169fe3a-4dc8-43bd-9017-dc5124b327aa","Type":"ContainerStarted","Data":"88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa"}
Jul 30 19:50:34.414606 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:34.414137    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c" event={"ID":"f169fe3a-4dc8-43bd-9017-dc5124b327aa","Type":"ContainerStarted","Data":"2fb54e482cb83e587100438fb8680cae212f764c8c393a8e6943790dec312d5d"}
Jul 30 19:50:34.431368 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:34.431316    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c" podStartSLOduration=1.431301761 podStartE2EDuration="1.431301761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:50:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:50:34.430587601 +0000 UTC m=+1554.191342859" watchObservedRunningTime="2026-07-30 19:50:34.431301761 +0000 UTC m=+1554.192056996"
Jul 30 19:50:39.960446 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:39.960411    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:50:43.586164 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:43.586132    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:43.586164 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:43.586167    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:43.598200 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:43.598175    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:50:44.466466 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:50:44.466433    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:51:22.150760 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:22.150729    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"]
Jul 30 19:51:22.831692 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:22.831662    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"]
Jul 30 19:51:24.809151 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:24.809121    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:51:37.169192 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.169131    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t" podUID="887ec1b5-c007-4109-9742-0ded5cdf86d0" containerName="main" containerID="cri-o://5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e" gracePeriod=45
Jul 30 19:51:37.408416 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.408391    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:51:37.455707 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.455656    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qng9g\" (UniqueName: \"kubernetes.io/projected/887ec1b5-c007-4109-9742-0ded5cdf86d0-kube-api-access-qng9g\") pod \"887ec1b5-c007-4109-9742-0ded5cdf86d0\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") "
Jul 30 19:51:37.455707 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.455694    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/887ec1b5-c007-4109-9742-0ded5cdf86d0-tls-certs\") pod \"887ec1b5-c007-4109-9742-0ded5cdf86d0\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") "
Jul 30 19:51:37.455867 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.455727    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-home\") pod \"887ec1b5-c007-4109-9742-0ded5cdf86d0\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") "
Jul 30 19:51:37.455867 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.455744    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-model-cache\") pod \"887ec1b5-c007-4109-9742-0ded5cdf86d0\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") "
Jul 30 19:51:37.455977 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.455878    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-dshm\") pod \"887ec1b5-c007-4109-9742-0ded5cdf86d0\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") "
Jul 30 19:51:37.455977 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.455945    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-tmp-dir\") pod \"887ec1b5-c007-4109-9742-0ded5cdf86d0\" (UID: \"887ec1b5-c007-4109-9742-0ded5cdf86d0\") "
Jul 30 19:51:37.456065 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.455976    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-home" (OuterVolumeSpecName: "home") pod "887ec1b5-c007-4109-9742-0ded5cdf86d0" (UID: "887ec1b5-c007-4109-9742-0ded5cdf86d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:51:37.456065 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.455992    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-model-cache" (OuterVolumeSpecName: "model-cache") pod "887ec1b5-c007-4109-9742-0ded5cdf86d0" (UID: "887ec1b5-c007-4109-9742-0ded5cdf86d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:51:37.456224 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.456206    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-home\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:37.456277 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.456231    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-model-cache\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:37.456277 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.456229    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "887ec1b5-c007-4109-9742-0ded5cdf86d0" (UID: "887ec1b5-c007-4109-9742-0ded5cdf86d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:51:37.457865 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.457836    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/887ec1b5-c007-4109-9742-0ded5cdf86d0-kube-api-access-qng9g" (OuterVolumeSpecName: "kube-api-access-qng9g") pod "887ec1b5-c007-4109-9742-0ded5cdf86d0" (UID: "887ec1b5-c007-4109-9742-0ded5cdf86d0"). InnerVolumeSpecName "kube-api-access-qng9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:51:37.457954 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.457874    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/887ec1b5-c007-4109-9742-0ded5cdf86d0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "887ec1b5-c007-4109-9742-0ded5cdf86d0" (UID: "887ec1b5-c007-4109-9742-0ded5cdf86d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:51:37.457954 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.457871    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-dshm" (OuterVolumeSpecName: "dshm") pod "887ec1b5-c007-4109-9742-0ded5cdf86d0" (UID: "887ec1b5-c007-4109-9742-0ded5cdf86d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:51:37.556669 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.556650    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-dshm\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:37.556669 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.556668    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/887ec1b5-c007-4109-9742-0ded5cdf86d0-tmp-dir\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:37.556792 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.556679    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qng9g\" (UniqueName: \"kubernetes.io/projected/887ec1b5-c007-4109-9742-0ded5cdf86d0-kube-api-access-qng9g\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:37.556792 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.556688    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/887ec1b5-c007-4109-9742-0ded5cdf86d0-tls-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:37.661723 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.661695    2577 generic.go:356] "Generic (PLEG): container finished" podID="887ec1b5-c007-4109-9742-0ded5cdf86d0" containerID="5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e" exitCode=0
Jul 30 19:51:37.661807 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.661733    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"
Jul 30 19:51:37.661807 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.661775    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t" event={"ID":"887ec1b5-c007-4109-9742-0ded5cdf86d0","Type":"ContainerDied","Data":"5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e"}
Jul 30 19:51:37.661879 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.661809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t" event={"ID":"887ec1b5-c007-4109-9742-0ded5cdf86d0","Type":"ContainerDied","Data":"f3b3248c3e011bd8f344ae6930843165a165b9a470bf7f7b4d3779c6b3ffcee5"}
Jul 30 19:51:37.661879 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.661829    2577 scope.go:117] "RemoveContainer" containerID="5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e"
Jul 30 19:51:37.670261 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.670246    2577 scope.go:117] "RemoveContainer" containerID="5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e"
Jul 30 19:51:37.670469 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:51:37.670444    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e\": container with ID starting with 5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e not found: ID does not exist" containerID="5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e"
Jul 30 19:51:37.670529 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.670469    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e"} err="failed to get container status \"5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e\": rpc error: code = NotFound desc = could not find container \"5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e\": container with ID starting with 5bc1c4b5b24474ca7e75db060ae6bf99f3dd089fac6630fb7debb33a09b7534e not found: ID does not exist"
Jul 30 19:51:37.681655 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.681634    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"]
Jul 30 19:51:37.685174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.685145    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-3dbce65f-kserve-6d5665ff5-5pb5t"]
Jul 30 19:51:37.850061 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:37.850013    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c" podUID="f169fe3a-4dc8-43bd-9017-dc5124b327aa" containerName="main" containerID="cri-o://88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa" gracePeriod=45
Jul 30 19:51:38.072175 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.072152    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:51:38.161783 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.161711    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-model-cache\") pod \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") "
Jul 30 19:51:38.161783 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.161751    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-dshm\") pod \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") "
Jul 30 19:51:38.161783 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.161775    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tls-certs\") pod \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") "
Jul 30 19:51:38.162003 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.161800    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-home\") pod \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") "
Jul 30 19:51:38.162003 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.161851    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tmp-dir\") pod \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") "
Jul 30 19:51:38.162003 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.161877    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqv4k\" (UniqueName: \"kubernetes.io/projected/f169fe3a-4dc8-43bd-9017-dc5124b327aa-kube-api-access-rqv4k\") pod \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\" (UID: \"f169fe3a-4dc8-43bd-9017-dc5124b327aa\") "
Jul 30 19:51:38.162170 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.162007    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "f169fe3a-4dc8-43bd-9017-dc5124b327aa" (UID: "f169fe3a-4dc8-43bd-9017-dc5124b327aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:51:38.162170 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.162122    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-home" (OuterVolumeSpecName: "home") pod "f169fe3a-4dc8-43bd-9017-dc5124b327aa" (UID: "f169fe3a-4dc8-43bd-9017-dc5124b327aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:51:38.162170 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.162132    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f169fe3a-4dc8-43bd-9017-dc5124b327aa" (UID: "f169fe3a-4dc8-43bd-9017-dc5124b327aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:51:38.162170 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.162154    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-model-cache\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:38.163831 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.163808    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-dshm" (OuterVolumeSpecName: "dshm") pod "f169fe3a-4dc8-43bd-9017-dc5124b327aa" (UID: "f169fe3a-4dc8-43bd-9017-dc5124b327aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:51:38.163831 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.163814    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f169fe3a-4dc8-43bd-9017-dc5124b327aa" (UID: "f169fe3a-4dc8-43bd-9017-dc5124b327aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:51:38.163953 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.163862    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f169fe3a-4dc8-43bd-9017-dc5124b327aa-kube-api-access-rqv4k" (OuterVolumeSpecName: "kube-api-access-rqv4k") pod "f169fe3a-4dc8-43bd-9017-dc5124b327aa" (UID: "f169fe3a-4dc8-43bd-9017-dc5124b327aa"). InnerVolumeSpecName "kube-api-access-rqv4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:51:38.263249 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.263228    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-dshm\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:38.263249 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.263248    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tls-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:38.263539 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.263257    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-home\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:38.263539 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.263264    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f169fe3a-4dc8-43bd-9017-dc5124b327aa-tmp-dir\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:38.263539 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.263272    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqv4k\" (UniqueName: \"kubernetes.io/projected/f169fe3a-4dc8-43bd-9017-dc5124b327aa-kube-api-access-rqv4k\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:51:38.565593 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.565572    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:51:38.666519 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.666492    2577 generic.go:356] "Generic (PLEG): container finished" podID="f169fe3a-4dc8-43bd-9017-dc5124b327aa" containerID="88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa" exitCode=0
Jul 30 19:51:38.666624 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.666533    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"
Jul 30 19:51:38.666624 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.666576    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c" event={"ID":"f169fe3a-4dc8-43bd-9017-dc5124b327aa","Type":"ContainerDied","Data":"88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa"}
Jul 30 19:51:38.666624 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.666614    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c" event={"ID":"f169fe3a-4dc8-43bd-9017-dc5124b327aa","Type":"ContainerDied","Data":"2fb54e482cb83e587100438fb8680cae212f764c8c393a8e6943790dec312d5d"}
Jul 30 19:51:38.666762 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.666637    2577 scope.go:117] "RemoveContainer" containerID="88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa"
Jul 30 19:51:38.674588 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.674572    2577 scope.go:117] "RemoveContainer" containerID="88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa"
Jul 30 19:51:38.674849 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:51:38.674833    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa\": container with ID starting with 88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa not found: ID does not exist" containerID="88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa"
Jul 30 19:51:38.674896 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.674855    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa"} err="failed to get container status \"88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa\": rpc error: code = NotFound desc = could not find container \"88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa\": container with ID starting with 88551c04b0fbe51bb01d4b4f340436cf3d4a60b51a4f7d7561b391022953bbfa not found: ID does not exist"
Jul 30 19:51:38.685883 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.685861    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"]
Jul 30 19:51:38.691965 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.691943    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-3dbce65f-kserve-d5dc94dc9-nx44c"]
Jul 30 19:51:38.787933 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.787904    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="887ec1b5-c007-4109-9742-0ded5cdf86d0" path="/var/lib/kubelet/pods/887ec1b5-c007-4109-9742-0ded5cdf86d0/volumes"
Jul 30 19:51:38.788289 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:38.788275    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f169fe3a-4dc8-43bd-9017-dc5124b327aa" path="/var/lib/kubelet/pods/f169fe3a-4dc8-43bd-9017-dc5124b327aa/volumes"
Jul 30 19:51:47.860402 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:51:47.860361    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:52:10.666462 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:52:10.666435    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:52:23.772112 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:52:23.772062    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:52:35.166585 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:52:35.166551    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:52:43.377408 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:52:43.377367    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:52:52.464020 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:52:52.463986    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:53:02.567939 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:53:02.567861    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:53:15.570532 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:53:15.570491    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:53:24.363469 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:53:24.363437    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:53:33.968246 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:53:33.968203    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:53:44.962148 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:53:44.962108    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:53:57.256199 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:53:57.256164    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:54:06.267328 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:54:06.267295    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:54:28.959180 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:54:28.959153    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:54:40.772025 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:54:40.771992    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:54:51.166149 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:54:51.166117    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:55:11.464514 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:55:11.464481    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:55:20.864671 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:55:20.864639    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:55:33.567961 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:55:33.567929    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:55:47.465685 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:55:47.465650    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:55:56.763943 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:55:56.763906    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:56:08.568267 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:08.568184    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:56:18.862051 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:18.862018    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:56:44.554283 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.554250    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"]
Jul 30 19:56:44.556748 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.554561    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f169fe3a-4dc8-43bd-9017-dc5124b327aa" containerName="main"
Jul 30 19:56:44.556748 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.554575    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f169fe3a-4dc8-43bd-9017-dc5124b327aa" containerName="main"
Jul 30 19:56:44.556748 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.554597    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="887ec1b5-c007-4109-9742-0ded5cdf86d0" containerName="main"
Jul 30 19:56:44.556748 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.554602    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="887ec1b5-c007-4109-9742-0ded5cdf86d0" containerName="main"
Jul 30 19:56:44.556748 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.554652    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="887ec1b5-c007-4109-9742-0ded5cdf86d0" containerName="main"
Jul 30 19:56:44.556748 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.554662    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f169fe3a-4dc8-43bd-9017-dc5124b327aa" containerName="main"
Jul 30 19:56:44.557737 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.557714    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.560250 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.560225    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 19:56:44.560250 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.560244    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-57cec089-gw-openshift-default-dockercfg-x42k5\""
Jul 30 19:56:44.572755 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.572732    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"]
Jul 30 19:56:44.653496 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.653474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-token\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.653600 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.653507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-credential-socket\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.653600 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.653533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-certs\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.653600 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.653551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-envoy\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.653600 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.653590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7klwv\" (UniqueName: \"kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-kube-api-access-7klwv\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.653735 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.653608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istiod-ca-cert\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.653735 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.653630    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-podinfo\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.653735 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.653707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-data\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.653831 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.653743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-socket\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.754765 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.754740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7klwv\" (UniqueName: \"kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-kube-api-access-7klwv\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.754854 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.754769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istiod-ca-cert\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.754854 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.754786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-podinfo\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.754854 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.754845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-data\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.754976 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.754882    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-socket\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.754976 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.754946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-token\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.755095 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.754990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-credential-socket\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.755095 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.755016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-certs\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.755095 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.755036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-envoy\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.755306 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.755283    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-data\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.755429 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.755409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-credential-socket\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.755483 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.755460    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-socket\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.755550 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.755527    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istiod-ca-cert\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.755586 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.755552    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-certs\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.757170 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.757151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-envoy\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.757249 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.757199    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-podinfo\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.762885 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.762861    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-token\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.763813 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.763794    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7klwv\" (UniqueName: \"kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-kube-api-access-7klwv\") pod \"e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") " pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.868202 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.868152    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:44.994724 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.994702    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"]
Jul 30 19:56:44.996913 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:56:44.996884    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4eda2a6_002a_4d1b_96b4_f2a64ebc9128.slice/crio-1b567217af5f648428e87046eea0888623c661ecf1a19832e5a9314b8b0776f8 WatchSource:0}: Error finding container 1b567217af5f648428e87046eea0888623c661ecf1a19832e5a9314b8b0776f8: Status 404 returned error can't find the container with id 1b567217af5f648428e87046eea0888623c661ecf1a19832e5a9314b8b0776f8
Jul 30 19:56:44.998931 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:44.998914    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 19:56:45.811903 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:45.811871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" event={"ID":"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128","Type":"ContainerStarted","Data":"1b567217af5f648428e87046eea0888623c661ecf1a19832e5a9314b8b0776f8"}
Jul 30 19:56:47.068405 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:47.068363    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:56:47.740526 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:47.740492    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:56:47.740604 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:47.740581    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:56:47.740644 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:47.740608    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:56:47.822313 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:47.822284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" event={"ID":"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128","Type":"ContainerStarted","Data":"7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e"}
Jul 30 19:56:47.843217 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:47.843168    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" podStartSLOduration=1.101966811 podStartE2EDuration="3.843148241s" totalImagesPullingTime="2.74118143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:56:47.841856329 +0000 UTC m=+1927.602611578" watchObservedRunningTime="2026-07-30 19:56:47.843148241 +0000 UTC m=+1927.603903497"
Jul 30 19:56:47.868656 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:47.868622    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:48.873415 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:48.873385    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:49.829609 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:49.829578    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:49.830488 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:49.830467    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:56:59.866366 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:56:59.866331    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:57:01.877746 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:01.877715    2577 generic.go:356] "Generic (PLEG): container finished" podID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerID="7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e" exitCode=0
Jul 30 19:57:01.878194 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:01.877789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" event={"ID":"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128","Type":"ContainerDied","Data":"7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e"}
Jul 30 19:57:01.878194 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:01.878133    2577 scope.go:117] "RemoveContainer" containerID="7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e"
Jul 30 19:57:01.879995 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:01.879966    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:57:01.880052 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:01.880034    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:57:01.880104 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:01.880064    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:57:02.883238 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:02.883204    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" event={"ID":"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128","Type":"ContainerStarted","Data":"eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c"}
Jul 30 19:57:03.869339 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:03.869301    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:57:03.870249 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:03.870220    2577 patch_prober.go:28] interesting pod/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 30 19:57:03.870351 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:03.870273    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 30 19:57:04.830462 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:04.830426    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:57:04.868897 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:04.868867    2577 patch_prober.go:28] interesting pod/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 30 19:57:04.869034 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:04.868918    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 30 19:57:05.868531 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:05.868493    2577 patch_prober.go:28] interesting pod/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Jul 30 19:57:05.868907 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:05.868585    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Jul 30 19:57:06.872091 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:06.872045    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:57:06.873145 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:06.873124    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:57:21.157011 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:21.156981    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"]
Jul 30 19:57:21.157519 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:21.157187    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerName="istio-proxy" containerID="cri-o://eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c" gracePeriod=30
Jul 30 19:57:21.875003 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:21.874968    2577 patch_prober.go:28] interesting pod/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 19:57:21.875177 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:21.875019    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 19:57:24.869859 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:24.869825    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:57:26.400914 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.400890    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:57:26.460914 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.460858    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-credential-socket\") pod \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") "
Jul 30 19:57:26.460914 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.460890    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-podinfo\") pod \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") "
Jul 30 19:57:26.460914 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.460908    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-socket\") pod \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") "
Jul 30 19:57:26.461190 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.460931    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7klwv\" (UniqueName: \"kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-kube-api-access-7klwv\") pod \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") "
Jul 30 19:57:26.461190 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.460951    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-token\") pod \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") "
Jul 30 19:57:26.461190 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.460987    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-envoy\") pod \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") "
Jul 30 19:57:26.461190 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.461026    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istiod-ca-cert\") pod \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") "
Jul 30 19:57:26.461190 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.461049    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-certs\") pod \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") "
Jul 30 19:57:26.461190 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.461095    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-data\") pod \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\" (UID: \"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128\") "
Jul 30 19:57:26.461190 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.461155    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" (UID: "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:57:26.461492 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.461236    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" (UID: "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:57:26.461492 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.461330    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-credential-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:57:26.461492 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.461380    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:57:26.461492 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.461411    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" (UID: "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:57:26.461665 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.461646    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" (UID: "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:57:26.462091 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.462057    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" (UID: "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:57:26.463366 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.463336    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" (UID: "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:57:26.463458 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.463406    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" (UID: "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:57:26.463543 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.463528    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" (UID: "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 19:57:26.463585 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.463558    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-kube-api-access-7klwv" (OuterVolumeSpecName: "kube-api-access-7klwv") pod "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" (UID: "e4eda2a6-002a-4d1b-96b4-f2a64ebc9128"). InnerVolumeSpecName "kube-api-access-7klwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:57:26.562288 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.562270    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-data\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:57:26.562288 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.562290    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-podinfo\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:57:26.562420 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.562300    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7klwv\" (UniqueName: \"kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-kube-api-access-7klwv\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:57:26.562420 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.562310    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:57:26.562420 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.562318    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istio-envoy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:57:26.562420 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.562325    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-istiod-ca-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:57:26.562420 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.562333    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128-workload-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:57:26.982812 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.982786    2577 generic.go:356] "Generic (PLEG): container finished" podID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerID="eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c" exitCode=0
Jul 30 19:57:26.982933 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.982841    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"
Jul 30 19:57:26.982933 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.982852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" event={"ID":"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128","Type":"ContainerDied","Data":"eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c"}
Jul 30 19:57:26.982933 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.982909    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt" event={"ID":"e4eda2a6-002a-4d1b-96b4-f2a64ebc9128","Type":"ContainerDied","Data":"1b567217af5f648428e87046eea0888623c661ecf1a19832e5a9314b8b0776f8"}
Jul 30 19:57:26.983114 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.982943    2577 scope.go:117] "RemoveContainer" containerID="eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c"
Jul 30 19:57:26.991023 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.991002    2577 scope.go:117] "RemoveContainer" containerID="7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e"
Jul 30 19:57:26.998348 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.998327    2577 scope.go:117] "RemoveContainer" containerID="eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c"
Jul 30 19:57:26.998603 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:57:26.998584    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c\": container with ID starting with eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c not found: ID does not exist" containerID="eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c"
Jul 30 19:57:26.998651 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.998610    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c"} err="failed to get container status \"eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c\": rpc error: code = NotFound desc = could not find container \"eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c\": container with ID starting with eb955d5be4b50b5218e6732b2d162ba8a10b5766fb43503eb9d80c28591d592c not found: ID does not exist"
Jul 30 19:57:26.998651 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.998628    2577 scope.go:117] "RemoveContainer" containerID="7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e"
Jul 30 19:57:26.998872 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:57:26.998852    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e\": container with ID starting with 7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e not found: ID does not exist" containerID="7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e"
Jul 30 19:57:26.998923 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:26.998881    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e"} err="failed to get container status \"7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e\": rpc error: code = NotFound desc = could not find container \"7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e\": container with ID starting with 7d875b54fd0f251cd3c7dcf615536ab5ba437f6f7d01552c07412fc874e2472e not found: ID does not exist"
Jul 30 19:57:27.002486 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:27.002463    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"]
Jul 30 19:57:27.006294 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:27.006274    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-57cec089-gw-openshift-default-57786dfbd8-qcfvt"]
Jul 30 19:57:28.784016 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:28.783983    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" path="/var/lib/kubelet/pods/e4eda2a6-002a-4d1b-96b4-f2a64ebc9128/volumes"
Jul 30 19:57:36.362371 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:36.362294    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:57:57.953838 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:57.953805    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"]
Jul 30 19:57:57.954257 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:57.954061    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerName="istio-proxy"
Jul 30 19:57:57.954257 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:57.954092    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerName="istio-proxy"
Jul 30 19:57:57.954257 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:57.954106    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerName="istio-proxy"
Jul 30 19:57:57.954257 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:57.954112    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerName="istio-proxy"
Jul 30 19:57:57.954257 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:57.954157    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerName="istio-proxy"
Jul 30 19:57:57.954257 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:57.954165    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4eda2a6-002a-4d1b-96b4-f2a64ebc9128" containerName="istio-proxy"
Jul 30 19:57:57.956954 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:57.956933    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:57.959233 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:57.959212    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 19:57:57.959750 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:57.959734    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-b83b31b4-openshift-default-dockercfg-pt6rk\""
Jul 30 19:57:57.966829 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:57.966809    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"]
Jul 30 19:57:58.070532 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.070507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-podinfo\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.070642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.070547    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-envoy\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.070642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.070569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-token\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.070642 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.070591    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istiod-ca-cert\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.070763 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.070669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-data\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.070763 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.070726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-credential-socket\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.070834 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.070774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-socket\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.070834 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.070798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-certs\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.070834 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.070829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9lnj\" (UniqueName: \"kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-kube-api-access-j9lnj\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171233 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171204    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-data\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171342 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-credential-socket\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171342 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-socket\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171342 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-certs\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171342 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9lnj\" (UniqueName: \"kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-kube-api-access-j9lnj\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171342 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-podinfo\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171602 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-envoy\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171602 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-token\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171602 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istiod-ca-cert\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171793 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171641    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-credential-socket\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171793 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171653    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-data\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171915 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171845    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-certs\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.171915 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.171887    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-socket\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.172107 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.172088    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istiod-ca-cert\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.173618 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.173598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-envoy\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.173850 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.173829    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-podinfo\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.179174 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.179149    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-token\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.179276 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.179159    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9lnj\" (UniqueName: \"kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-kube-api-access-j9lnj\") pod \"e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") " pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.267807 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.267745    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:58.397927 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.397886    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"]
Jul 30 19:57:58.400818 ip-10-0-129-203 kubenswrapper[2577]: W0730 19:57:58.400782    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a13be3b_ec52_4d4a_ab59_a378443af9ec.slice/crio-dd7ac5c82f7454c7ca4e896caf071ef4c56e8d9d5cd2b03a5f403454dcde8a28 WatchSource:0}: Error finding container dd7ac5c82f7454c7ca4e896caf071ef4c56e8d9d5cd2b03a5f403454dcde8a28: Status 404 returned error can't find the container with id dd7ac5c82f7454c7ca4e896caf071ef4c56e8d9d5cd2b03a5f403454dcde8a28
Jul 30 19:57:58.402976 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.402937    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:57:58.403041 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.403025    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:57:58.403104 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:58.403092    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:57:59.102920 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:59.102885    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" event={"ID":"0a13be3b-ec52-4d4a-ab59-a378443af9ec","Type":"ContainerStarted","Data":"4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6"}
Jul 30 19:57:59.103321 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:59.102937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" event={"ID":"0a13be3b-ec52-4d4a-ab59-a378443af9ec","Type":"ContainerStarted","Data":"dd7ac5c82f7454c7ca4e896caf071ef4c56e8d9d5cd2b03a5f403454dcde8a28"}
Jul 30 19:57:59.123533 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:59.123489    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" podStartSLOduration=2.123474984 podStartE2EDuration="2.123474984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:57:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:57:59.122943615 +0000 UTC m=+1998.883698884" watchObservedRunningTime="2026-07-30 19:57:59.123474984 +0000 UTC m=+1998.884230241"
Jul 30 19:57:59.268445 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:59.268417    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:59.272905 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:59.272882    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:57:59.859455 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:57:59.859423    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:58:00.106647 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:00.106615    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:58:00.107593 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:00.107572    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:58:09.065047 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:09.065010    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:58:13.160289 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:13.160258    2577 generic.go:356] "Generic (PLEG): container finished" podID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerID="4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6" exitCode=0
Jul 30 19:58:13.160668 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:13.160313    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" event={"ID":"0a13be3b-ec52-4d4a-ab59-a378443af9ec","Type":"ContainerDied","Data":"4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6"}
Jul 30 19:58:13.160668 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:13.160611    2577 scope.go:117] "RemoveContainer" containerID="4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6"
Jul 30 19:58:13.161525 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:13.161496    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:58:13.161576 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:13.161561    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:58:13.161611 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:13.161592    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 19:58:14.167256 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:14.167222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" event={"ID":"0a13be3b-ec52-4d4a-ab59-a378443af9ec","Type":"ContainerStarted","Data":"a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c"}
Jul 30 19:58:14.268543 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:14.268491    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:58:14.269529 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:14.269501    2577 patch_prober.go:28] interesting pod/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 30 19:58:14.269641 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:14.269567    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 30 19:58:15.107056 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:15.107022    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:58:15.268933 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:15.268902    2577 patch_prober.go:28] interesting pod/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 30 19:58:15.269319 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:15.268966    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 30 19:58:16.268643 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:16.268608    2577 patch_prober.go:28] interesting pod/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 30 19:58:16.268816 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:16.268671    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 30 19:58:17.268967 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:17.268930    2577 patch_prober.go:28] interesting pod/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 30 19:58:17.269370 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:17.268983    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 30 19:58:18.268509 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:18.268455    2577 patch_prober.go:28] interesting pod/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Jul 30 19:58:18.268686 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:18.268531    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Jul 30 19:58:19.271776 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:19.271699    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:58:19.272645 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:19.272622    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:58:32.279260 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:32.279231    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"]
Jul 30 19:58:32.279705 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:32.279437    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy" containerID="cri-o://a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c" gracePeriod=30
Jul 30 19:58:34.274771 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:34.274738    2577 patch_prober.go:28] interesting pod/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 19:58:34.275155 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:34.274791    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 19:58:35.870578 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:35.870547    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 19:58:37.528214 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.528186    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:58:37.650868 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.650831    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-token\") pod \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") "
Jul 30 19:58:37.651047 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.650881    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-credential-socket\") pod \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") "
Jul 30 19:58:37.651047 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.650912    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istiod-ca-cert\") pod \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") "
Jul 30 19:58:37.651047 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.650939    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-data\") pod \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") "
Jul 30 19:58:37.651047 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.650968    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9lnj\" (UniqueName: \"kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-kube-api-access-j9lnj\") pod \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") "
Jul 30 19:58:37.651047 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.650998    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-envoy\") pod \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") "
Jul 30 19:58:37.651047 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.651025    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-certs\") pod \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") "
Jul 30 19:58:37.651434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.651131    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-podinfo\") pod \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") "
Jul 30 19:58:37.651434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.651141    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a13be3b-ec52-4d4a-ab59-a378443af9ec" (UID: "0a13be3b-ec52-4d4a-ab59-a378443af9ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:58:37.651434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.651162    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-socket\") pod \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\" (UID: \"0a13be3b-ec52-4d4a-ab59-a378443af9ec\") "
Jul 30 19:58:37.651434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.651343    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a13be3b-ec52-4d4a-ab59-a378443af9ec" (UID: "0a13be3b-ec52-4d4a-ab59-a378443af9ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:58:37.651434 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.651404    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-credential-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:58:37.651701 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.651575    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a13be3b-ec52-4d4a-ab59-a378443af9ec" (UID: "0a13be3b-ec52-4d4a-ab59-a378443af9ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:58:37.651951 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.651907    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a13be3b-ec52-4d4a-ab59-a378443af9ec" (UID: "0a13be3b-ec52-4d4a-ab59-a378443af9ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:58:37.652108 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.652035    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a13be3b-ec52-4d4a-ab59-a378443af9ec" (UID: "0a13be3b-ec52-4d4a-ab59-a378443af9ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:58:37.653543 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.653507    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a13be3b-ec52-4d4a-ab59-a378443af9ec" (UID: "0a13be3b-ec52-4d4a-ab59-a378443af9ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:58:37.653746 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.653720    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-kube-api-access-j9lnj" (OuterVolumeSpecName: "kube-api-access-j9lnj") pod "0a13be3b-ec52-4d4a-ab59-a378443af9ec" (UID: "0a13be3b-ec52-4d4a-ab59-a378443af9ec"). InnerVolumeSpecName "kube-api-access-j9lnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:58:37.654099 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.654053    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a13be3b-ec52-4d4a-ab59-a378443af9ec" (UID: "0a13be3b-ec52-4d4a-ab59-a378443af9ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 19:58:37.654183 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.654163    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a13be3b-ec52-4d4a-ab59-a378443af9ec" (UID: "0a13be3b-ec52-4d4a-ab59-a378443af9ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:58:37.752766 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.752731    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-podinfo\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:58:37.752766 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.752760    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:58:37.752766 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.752771    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:58:37.752982 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.752780    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istiod-ca-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:58:37.752982 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.752788    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-data\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:58:37.752982 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.752796    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9lnj\" (UniqueName: \"kubernetes.io/projected/0a13be3b-ec52-4d4a-ab59-a378443af9ec-kube-api-access-j9lnj\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:58:37.752982 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.752804    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-istio-envoy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:58:37.752982 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:37.752812    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a13be3b-ec52-4d4a-ab59-a378443af9ec-workload-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 19:58:38.267372 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.267337    2577 generic.go:356] "Generic (PLEG): container finished" podID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerID="a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c" exitCode=0
Jul 30 19:58:38.267550 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.267386    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"
Jul 30 19:58:38.267550 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.267416    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" event={"ID":"0a13be3b-ec52-4d4a-ab59-a378443af9ec","Type":"ContainerDied","Data":"a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c"}
Jul 30 19:58:38.267550 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.267470    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x" event={"ID":"0a13be3b-ec52-4d4a-ab59-a378443af9ec","Type":"ContainerDied","Data":"dd7ac5c82f7454c7ca4e896caf071ef4c56e8d9d5cd2b03a5f403454dcde8a28"}
Jul 30 19:58:38.267550 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.267492    2577 scope.go:117] "RemoveContainer" containerID="a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c"
Jul 30 19:58:38.276362 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.276344    2577 scope.go:117] "RemoveContainer" containerID="4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6"
Jul 30 19:58:38.283370 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.283354    2577 scope.go:117] "RemoveContainer" containerID="a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c"
Jul 30 19:58:38.283613 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:58:38.283590    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c\": container with ID starting with a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c not found: ID does not exist" containerID="a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c"
Jul 30 19:58:38.283723 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.283620    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c"} err="failed to get container status \"a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c\": rpc error: code = NotFound desc = could not find container \"a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c\": container with ID starting with a3faefeb0231a9d3c6cd355ef7b78fa0d34f1445b799befbfabed45445d50e4c not found: ID does not exist"
Jul 30 19:58:38.283723 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.283638    2577 scope.go:117] "RemoveContainer" containerID="4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6"
Jul 30 19:58:38.283895 ip-10-0-129-203 kubenswrapper[2577]: E0730 19:58:38.283875    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6\": container with ID starting with 4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6 not found: ID does not exist" containerID="4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6"
Jul 30 19:58:38.283940 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.283894    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6"} err="failed to get container status \"4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6\": rpc error: code = NotFound desc = could not find container \"4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6\": container with ID starting with 4b3196b2ed20507d74f0e85c906465ba2c6220f98f349bb184801b6e27b6abc6 not found: ID does not exist"
Jul 30 19:58:38.288511 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.288489    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"]
Jul 30 19:58:38.291944 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.291924    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-b83b31b4-openshift-default-6c99bb5dd4-sqm7x"]
Jul 30 19:58:38.784693 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:38.784664    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" path="/var/lib/kubelet/pods/0a13be3b-ec52-4d4a-ab59-a378443af9ec/volumes"
Jul 30 19:58:58.957424 ip-10-0-129-203 kubenswrapper[2577]: I0730 19:58:58.957390    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:00:26.030571 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:26.030532    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:00:36.678695 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:36.678610    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:00:41.829450 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.829419    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"]
Jul 30 20:00:41.829827 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.829710    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy"
Jul 30 20:00:41.829827 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.829722    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy"
Jul 30 20:00:41.829827 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.829769    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy"
Jul 30 20:00:41.829827 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.829777    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy"
Jul 30 20:00:41.832529 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.832512    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.834757 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.834735    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-361f6a62-openshift-default-dockercfg-sq6zh\""
Jul 30 20:00:41.834908 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.834762    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 20:00:41.843936 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.843910    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"]
Jul 30 20:00:41.896028 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.895999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf6cr\" (UniqueName: \"kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-kube-api-access-kf6cr\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.896165 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.896039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-credential-socket\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.896165 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.896090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-data\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.896165 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.896128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-token\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.896281 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.896184    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istiod-ca-cert\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.896281 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.896216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-envoy\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.896281 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.896268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-certs\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.896377 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.896287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-socket\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.896377 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.896307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-podinfo\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997315 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-certs\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997414 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-socket\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997414 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-podinfo\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997414 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf6cr\" (UniqueName: \"kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-kube-api-access-kf6cr\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997414 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-credential-socket\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997414 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997406    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-data\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997668 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-token\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997724 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997685    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istiod-ca-cert\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997724 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-certs\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997828 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-envoy\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997828 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-data\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997934 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997837    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-credential-socket\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.997989 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.997944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-socket\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.998305 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.998287    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istiod-ca-cert\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.999858 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.999840    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-podinfo\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:41.999944 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:41.999856    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-envoy\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:42.010936 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:42.010915    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf6cr\" (UniqueName: \"kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-kube-api-access-kf6cr\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:42.018301 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:42.018279    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-token\") pod \"e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") " pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:42.144052 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:42.144004    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:42.286639 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:42.286612    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"]
Jul 30 20:00:42.287911 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:00:42.287883    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ce9451_46df_4008_ad8b_2c7241b6ebb7.slice/crio-20671bcf9bc6ea22b3c17b0f47ed5985b427b48fc905b42e359ec3c4f9e74710 WatchSource:0}: Error finding container 20671bcf9bc6ea22b3c17b0f47ed5985b427b48fc905b42e359ec3c4f9e74710: Status 404 returned error can't find the container with id 20671bcf9bc6ea22b3c17b0f47ed5985b427b48fc905b42e359ec3c4f9e74710
Jul 30 20:00:42.289905 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:42.289876    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:00:42.289967 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:42.289941    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:00:42.290019 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:42.289966    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:00:42.737227 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:42.737188    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" event={"ID":"85ce9451-46df-4008-ad8b-2c7241b6ebb7","Type":"ContainerStarted","Data":"0ab9f342078a3ddb6a1fc3d47892eda6e8e7739c1156a2fe21f7e2955a2b22d6"}
Jul 30 20:00:42.737399 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:42.737240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" event={"ID":"85ce9451-46df-4008-ad8b-2c7241b6ebb7","Type":"ContainerStarted","Data":"20671bcf9bc6ea22b3c17b0f47ed5985b427b48fc905b42e359ec3c4f9e74710"}
Jul 30 20:00:42.757234 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:42.757184    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" podStartSLOduration=1.757169619 podStartE2EDuration="1.757169619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:00:42.755752341 +0000 UTC m=+2162.516507601" watchObservedRunningTime="2026-07-30 20:00:42.757169619 +0000 UTC m=+2162.517924876"
Jul 30 20:00:43.144984 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:43.144960    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:43.149130 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:43.149106    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:43.740920 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:43.740886    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:43.741791 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:43.741773    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:00:50.287465 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:50.287436    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:00:58.742268 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:58.742235    2577 patch_prober.go:28] interesting pod/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Jul 30 20:00:58.742666 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:58.742303    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Jul 30 20:00:58.797879 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:58.797854    2577 generic.go:356] "Generic (PLEG): container finished" podID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerID="0ab9f342078a3ddb6a1fc3d47892eda6e8e7739c1156a2fe21f7e2955a2b22d6" exitCode=0
Jul 30 20:00:58.798006 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:58.797908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" event={"ID":"85ce9451-46df-4008-ad8b-2c7241b6ebb7","Type":"ContainerDied","Data":"0ab9f342078a3ddb6a1fc3d47892eda6e8e7739c1156a2fe21f7e2955a2b22d6"}
Jul 30 20:00:58.798199 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:58.798187    2577 scope.go:117] "RemoveContainer" containerID="0ab9f342078a3ddb6a1fc3d47892eda6e8e7739c1156a2fe21f7e2955a2b22d6"
Jul 30 20:00:58.799090 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:58.799050    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:00:58.799175 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:58.799131    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:00:58.799175 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:58.799164    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:00:59.803260 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:00:59.803224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" event={"ID":"85ce9451-46df-4008-ad8b-2c7241b6ebb7","Type":"ContainerStarted","Data":"e3fe37b668ea0855cc1c9c9217483f01282fad10b27fbee432adcd640af84591"}
Jul 30 20:01:00.145186 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:00.145083    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:01:00.145361 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:00.145220    2577 patch_prober.go:28] interesting pod/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Jul 30 20:01:00.145361 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:00.145264    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Jul 30 20:01:01.144912 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:01.144871    2577 patch_prober.go:28] interesting pod/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Jul 30 20:01:01.145458 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:01.144938    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Jul 30 20:01:02.144443 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:02.144404    2577 patch_prober.go:28] interesting pod/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Jul 30 20:01:02.144706 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:02.144676    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Jul 30 20:01:03.144494 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:03.144446    2577 patch_prober.go:28] interesting pod/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Jul 30 20:01:03.144986 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:03.144516    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Jul 30 20:01:04.149006 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:04.148973    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:01:04.149443 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:04.149315    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:01:04.150087 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:04.150046    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:01:30.063688 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:30.063651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:01:33.740132 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:33.740099    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"]
Jul 30 20:01:33.740594 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:33.740388    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy" containerID="cri-o://e3fe37b668ea0855cc1c9c9217483f01282fad10b27fbee432adcd640af84591" gracePeriod=30
Jul 30 20:01:34.151882 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:34.151834    2577 patch_prober.go:28] interesting pod/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 20:01:34.152051 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:34.151908    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 20:01:38.955996 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:38.955966    2577 generic.go:356] "Generic (PLEG): container finished" podID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerID="e3fe37b668ea0855cc1c9c9217483f01282fad10b27fbee432adcd640af84591" exitCode=0
Jul 30 20:01:38.956345 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:38.956039    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" event={"ID":"85ce9451-46df-4008-ad8b-2c7241b6ebb7","Type":"ContainerDied","Data":"e3fe37b668ea0855cc1c9c9217483f01282fad10b27fbee432adcd640af84591"}
Jul 30 20:01:38.956345 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:38.956128    2577 scope.go:117] "RemoveContainer" containerID="0ab9f342078a3ddb6a1fc3d47892eda6e8e7739c1156a2fe21f7e2955a2b22d6"
Jul 30 20:01:38.970928 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:38.970913    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:01:39.104584 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.104561    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-token\") pod \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") "
Jul 30 20:01:39.104747 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.104596    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-certs\") pod \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") "
Jul 30 20:01:39.104747 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.104625    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-data\") pod \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") "
Jul 30 20:01:39.104747 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.104646    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-podinfo\") pod \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") "
Jul 30 20:01:39.104905 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.104780    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istiod-ca-cert\") pod \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") "
Jul 30 20:01:39.104905 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.104822    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-envoy\") pod \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") "
Jul 30 20:01:39.104905 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.104855    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-credential-socket\") pod \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") "
Jul 30 20:01:39.104905 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.104901    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf6cr\" (UniqueName: \"kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-kube-api-access-kf6cr\") pod \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") "
Jul 30 20:01:39.105121 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.104921    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "85ce9451-46df-4008-ad8b-2c7241b6ebb7" (UID: "85ce9451-46df-4008-ad8b-2c7241b6ebb7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:01:39.105121 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.104993    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-socket\") pod \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\" (UID: \"85ce9451-46df-4008-ad8b-2c7241b6ebb7\") "
Jul 30 20:01:39.105284 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.105260    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:01:39.105284 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.105255    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "85ce9451-46df-4008-ad8b-2c7241b6ebb7" (UID: "85ce9451-46df-4008-ad8b-2c7241b6ebb7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:01:39.105534 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.105514    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-data" (OuterVolumeSpecName: "istio-data") pod "85ce9451-46df-4008-ad8b-2c7241b6ebb7" (UID: "85ce9451-46df-4008-ad8b-2c7241b6ebb7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:01:39.105600 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.105558    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "85ce9451-46df-4008-ad8b-2c7241b6ebb7" (UID: "85ce9451-46df-4008-ad8b-2c7241b6ebb7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:01:39.105637 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.105625    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "85ce9451-46df-4008-ad8b-2c7241b6ebb7" (UID: "85ce9451-46df-4008-ad8b-2c7241b6ebb7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:01:39.106946 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.106914    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-token" (OuterVolumeSpecName: "istio-token") pod "85ce9451-46df-4008-ad8b-2c7241b6ebb7" (UID: "85ce9451-46df-4008-ad8b-2c7241b6ebb7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:01:39.107047 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.106967    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-kube-api-access-kf6cr" (OuterVolumeSpecName: "kube-api-access-kf6cr") pod "85ce9451-46df-4008-ad8b-2c7241b6ebb7" (UID: "85ce9451-46df-4008-ad8b-2c7241b6ebb7"). InnerVolumeSpecName "kube-api-access-kf6cr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:01:39.107170 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.107151    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "85ce9451-46df-4008-ad8b-2c7241b6ebb7" (UID: "85ce9451-46df-4008-ad8b-2c7241b6ebb7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:01:39.107229 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.107171    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "85ce9451-46df-4008-ad8b-2c7241b6ebb7" (UID: "85ce9451-46df-4008-ad8b-2c7241b6ebb7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 20:01:39.206224 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.206201    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf6cr\" (UniqueName: \"kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-kube-api-access-kf6cr\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:01:39.206224 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.206221    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-workload-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:01:39.206336 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.206231    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:01:39.206336 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.206239    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-data\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:01:39.206336 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.206246    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-podinfo\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:01:39.206336 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.206255    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istiod-ca-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:01:39.206336 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.206262    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-istio-envoy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:01:39.206336 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.206270    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ce9451-46df-4008-ad8b-2c7241b6ebb7-credential-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:01:39.963253 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.963219    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"
Jul 30 20:01:39.963253 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.963228    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql" event={"ID":"85ce9451-46df-4008-ad8b-2c7241b6ebb7","Type":"ContainerDied","Data":"20671bcf9bc6ea22b3c17b0f47ed5985b427b48fc905b42e359ec3c4f9e74710"}
Jul 30 20:01:39.963724 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.963291    2577 scope.go:117] "RemoveContainer" containerID="e3fe37b668ea0855cc1c9c9217483f01282fad10b27fbee432adcd640af84591"
Jul 30 20:01:39.995453 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.995433    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"]
Jul 30 20:01:39.998882 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:39.998860    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-361f6a62-openshift-default-5b7b997c6-kgcql"]
Jul 30 20:01:40.784435 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:01:40.784399    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" path="/var/lib/kubelet/pods/85ce9451-46df-4008-ad8b-2c7241b6ebb7/volumes"
Jul 30 20:02:06.167761 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:02:06.167685    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:03:48.684737 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.684657    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"]
Jul 30 20:03:48.685240 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.685030    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy"
Jul 30 20:03:48.685240 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.685045    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy"
Jul 30 20:03:48.685240 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.685092    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy"
Jul 30 20:03:48.685240 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.685101    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a13be3b-ec52-4d4a-ab59-a378443af9ec" containerName="istio-proxy"
Jul 30 20:03:48.685240 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.685170    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy"
Jul 30 20:03:48.687153 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.687134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.689563 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.689538    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f6ff274e-openshift-default-dockercfg-tp947\""
Jul 30 20:03:48.689674 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.689549    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 20:03:48.698200 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.698169    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"]
Jul 30 20:03:48.776773 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.776750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-data\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.776897 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.776797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-socket\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.776897 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.776840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istiod-ca-cert\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.776897 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.776870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-credential-socket\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.777090 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.776895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-envoy\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.777090 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.776931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vkrb\" (UniqueName: \"kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-kube-api-access-6vkrb\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.777090 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.776966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-certs\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.777090 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.777014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-podinfo\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.777090 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.777052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-token\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878034 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-token\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878147 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878051    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-data\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878147 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-socket\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878147 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istiod-ca-cert\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878147 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878146    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-credential-socket\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878345 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-envoy\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878345 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vkrb\" (UniqueName: \"kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-kube-api-access-6vkrb\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878345 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-certs\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878470 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878406    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-podinfo\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878470 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878457    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-data\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878565 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878515    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-certs\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878766 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-credential-socket\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.878882 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.878797    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-socket\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.879188 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.879166    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istiod-ca-cert\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.880313 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.880292    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-envoy\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.880750 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.880732    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-podinfo\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.885906 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.885883    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-token\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.886002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.885889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vkrb\" (UniqueName: \"kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-kube-api-access-6vkrb\") pod \"e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") " pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:48.996971 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:48.996921    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:49.126137 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:49.126112    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"]
Jul 30 20:03:49.128467 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:03:49.128438    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b6e91bd_921b_4948_9f4f_ecf5bc5c67a8.slice/crio-e103436c807e7248f1b9b12463431d3cd8978e47d459c2f68b25107ab20cfad0 WatchSource:0}: Error finding container e103436c807e7248f1b9b12463431d3cd8978e47d459c2f68b25107ab20cfad0: Status 404 returned error can't find the container with id e103436c807e7248f1b9b12463431d3cd8978e47d459c2f68b25107ab20cfad0
Jul 30 20:03:49.130529 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:49.130499    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:03:49.130605 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:49.130567    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:03:49.130605 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:49.130594    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:03:49.442133 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:49.442095    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" event={"ID":"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8","Type":"ContainerStarted","Data":"c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20"}
Jul 30 20:03:49.442301 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:49.442147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" event={"ID":"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8","Type":"ContainerStarted","Data":"e103436c807e7248f1b9b12463431d3cd8978e47d459c2f68b25107ab20cfad0"}
Jul 30 20:03:49.471757 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:49.471702    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" podStartSLOduration=1.471686829 podStartE2EDuration="1.471686829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:03:49.470172332 +0000 UTC m=+2349.230927591" watchObservedRunningTime="2026-07-30 20:03:49.471686829 +0000 UTC m=+2349.232442086"
Jul 30 20:03:49.997241 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:49.997216    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:51.003815 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:51.003787    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:51.449456 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:51.449427    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:51.450378 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:51.450360    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:03:53.878850 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:03:53.878818    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:04:02.674018 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:02.673983    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:04:07.157668 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:07.157596    2577 patch_prober.go:28] interesting pod/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": read tcp 10.134.0.2:33154->10.134.0.30:15021: read: connection reset by peer" start-of-body=
Jul 30 20:04:07.158001 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:07.157700    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": read tcp 10.134.0.2:33154->10.134.0.30:15021: read: connection reset by peer"
Jul 30 20:04:07.512545 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:07.512467    2577 generic.go:356] "Generic (PLEG): container finished" podID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerID="c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20" exitCode=0
Jul 30 20:04:07.512545 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:07.512519    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" event={"ID":"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8","Type":"ContainerDied","Data":"c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20"}
Jul 30 20:04:07.512885 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:07.512867    2577 scope.go:117] "RemoveContainer" containerID="c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20"
Jul 30 20:04:07.513868 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:07.513837    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:04:07.513937 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:07.513908    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:04:07.514014 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:07.513941    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:04:08.518225 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:08.518190    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" event={"ID":"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8","Type":"ContainerStarted","Data":"f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47"}
Jul 30 20:04:08.997965 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:08.997938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:04:08.998146 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:08.998117    2577 patch_prober.go:28] interesting pod/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 30 20:04:08.998198 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:08.998160    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 30 20:04:09.998064 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:09.998031    2577 patch_prober.go:28] interesting pod/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 30 20:04:09.998446 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:09.998103    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 30 20:04:10.997717 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:10.997683    2577 patch_prober.go:28] interesting pod/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 30 20:04:10.997883 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:10.997747    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 30 20:04:11.998229 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:11.998193    2577 patch_prober.go:28] interesting pod/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 30 20:04:11.998626 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:11.998247    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 30 20:04:12.997281 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:12.997244    2577 patch_prober.go:28] interesting pod/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 30 20:04:12.997453 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:12.997299    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 30 20:04:13.998110 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:13.998051    2577 patch_prober.go:28] interesting pod/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 30 20:04:13.998512 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:13.998136    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 30 20:04:15.001346 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:15.001316    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:04:15.001765 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:15.001570    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:04:15.002329 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:15.002313    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:04:21.263032 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.262993    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"]
Jul 30 20:04:21.263621 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.263395    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy"
Jul 30 20:04:21.263621 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.263413    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy"
Jul 30 20:04:21.263621 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.263480    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="85ce9451-46df-4008-ad8b-2c7241b6ebb7" containerName="istio-proxy"
Jul 30 20:04:21.266294 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.266273    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.268793 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.268761    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f6ff274e\"/\"kube-root-ca.crt\""
Jul 30 20:04:21.268793 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.268779    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f6ff274e\"/\"openshift-service-ca.crt\""
Jul 30 20:04:21.269622 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.269605    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f6ff274e\"/\"default-dockercfg-whqvq\""
Jul 30 20:04:21.269715 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.269621    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f6ff274e\"/\"e2e-lifecycle-model-f6ff274e-kserve-self-signed-certs\""
Jul 30 20:04:21.275172 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.275153    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"]
Jul 30 20:04:21.306489 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.306464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6js2x\" (UniqueName: \"kubernetes.io/projected/43fea3ff-8a26-4965-96de-db6f708ab416-kube-api-access-6js2x\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.306593 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.306492    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43fea3ff-8a26-4965-96de-db6f708ab416-tls-certs\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.306593 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.306521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-home\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.306593 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.306546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-model-cache\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.306593 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.306562    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-tmp-dir\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.306593 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.306585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-dshm\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.407029 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.407003    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-tmp-dir\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.407177 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.407049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-dshm\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.407177 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.407125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6js2x\" (UniqueName: \"kubernetes.io/projected/43fea3ff-8a26-4965-96de-db6f708ab416-kube-api-access-6js2x\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.407177 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.407152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43fea3ff-8a26-4965-96de-db6f708ab416-tls-certs\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.407343 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.407198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-home\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.407343 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.407230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-model-cache\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.407448 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.407433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-tmp-dir\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.407559 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.407539    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-home\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.407618 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.407567    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-model-cache\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.409321 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.409302    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-dshm\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.409486 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.409469    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43fea3ff-8a26-4965-96de-db6f708ab416-tls-certs\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.415345 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.415323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6js2x\" (UniqueName: \"kubernetes.io/projected/43fea3ff-8a26-4965-96de-db6f708ab416-kube-api-access-6js2x\") pod \"e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") " pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.575979 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.575956    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:21.701145 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:21.701118    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"]
Jul 30 20:04:21.703378 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:04:21.703349    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43fea3ff_8a26_4965_96de_db6f708ab416.slice/crio-6503a73abe3190532d4bbd1a85a463850741479f3809f161d83d54385561d2c0 WatchSource:0}: Error finding container 6503a73abe3190532d4bbd1a85a463850741479f3809f161d83d54385561d2c0: Status 404 returned error can't find the container with id 6503a73abe3190532d4bbd1a85a463850741479f3809f161d83d54385561d2c0
Jul 30 20:04:22.576398 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:22.576363    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx" event={"ID":"43fea3ff-8a26-4965-96de-db6f708ab416","Type":"ContainerStarted","Data":"7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90"}
Jul 30 20:04:22.576398 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:22.576401    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx" event={"ID":"43fea3ff-8a26-4965-96de-db6f708ab416","Type":"ContainerStarted","Data":"6503a73abe3190532d4bbd1a85a463850741479f3809f161d83d54385561d2c0"}
Jul 30 20:04:22.593148 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:22.593102    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx" podStartSLOduration=1.593063144 podStartE2EDuration="1.593063144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:04:22.592230046 +0000 UTC m=+2382.352985303" watchObservedRunningTime="2026-07-30 20:04:22.593063144 +0000 UTC m=+2382.353818461"
Jul 30 20:04:24.363013 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:24.362981    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:04:31.577174 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:31.577134    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:31.577560 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:31.577192    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:31.589782 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:31.589756    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:31.620848 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:31.620823    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:04:36.873812 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:04:36.873779    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:05:09.769128 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:09.769031    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:05:18.464799 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:18.464766    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:05:19.891858 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:19.891826    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"]
Jul 30 20:05:34.910159 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:34.910120    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx" podUID="43fea3ff-8a26-4965-96de-db6f708ab416" containerName="main" containerID="cri-o://7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90" gracePeriod=45
Jul 30 20:05:35.142225 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.142202    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:05:35.301767 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.301734    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-dshm\") pod \"43fea3ff-8a26-4965-96de-db6f708ab416\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") "
Jul 30 20:05:35.301902 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.301782    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-tmp-dir\") pod \"43fea3ff-8a26-4965-96de-db6f708ab416\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") "
Jul 30 20:05:35.301902 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.301827    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6js2x\" (UniqueName: \"kubernetes.io/projected/43fea3ff-8a26-4965-96de-db6f708ab416-kube-api-access-6js2x\") pod \"43fea3ff-8a26-4965-96de-db6f708ab416\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") "
Jul 30 20:05:35.301902 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.301849    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-model-cache\") pod \"43fea3ff-8a26-4965-96de-db6f708ab416\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") "
Jul 30 20:05:35.301902 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.301889    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43fea3ff-8a26-4965-96de-db6f708ab416-tls-certs\") pod \"43fea3ff-8a26-4965-96de-db6f708ab416\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") "
Jul 30 20:05:35.302144 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.301918    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-home\") pod \"43fea3ff-8a26-4965-96de-db6f708ab416\" (UID: \"43fea3ff-8a26-4965-96de-db6f708ab416\") "
Jul 30 20:05:35.302144 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.302131    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43fea3ff-8a26-4965-96de-db6f708ab416" (UID: "43fea3ff-8a26-4965-96de-db6f708ab416"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:05:35.302249 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.302204    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-model-cache" (OuterVolumeSpecName: "model-cache") pod "43fea3ff-8a26-4965-96de-db6f708ab416" (UID: "43fea3ff-8a26-4965-96de-db6f708ab416"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:05:35.302292 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.302251    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-home" (OuterVolumeSpecName: "home") pod "43fea3ff-8a26-4965-96de-db6f708ab416" (UID: "43fea3ff-8a26-4965-96de-db6f708ab416"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:05:35.303852 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.303832    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-dshm" (OuterVolumeSpecName: "dshm") pod "43fea3ff-8a26-4965-96de-db6f708ab416" (UID: "43fea3ff-8a26-4965-96de-db6f708ab416"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:05:35.303928 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.303898    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43fea3ff-8a26-4965-96de-db6f708ab416-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43fea3ff-8a26-4965-96de-db6f708ab416" (UID: "43fea3ff-8a26-4965-96de-db6f708ab416"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:05:35.303996 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.303976    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43fea3ff-8a26-4965-96de-db6f708ab416-kube-api-access-6js2x" (OuterVolumeSpecName: "kube-api-access-6js2x") pod "43fea3ff-8a26-4965-96de-db6f708ab416" (UID: "43fea3ff-8a26-4965-96de-db6f708ab416"). InnerVolumeSpecName "kube-api-access-6js2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:05:35.402538 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.402515    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-dshm\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:05:35.402538 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.402536    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-tmp-dir\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:05:35.402678 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.402546    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6js2x\" (UniqueName: \"kubernetes.io/projected/43fea3ff-8a26-4965-96de-db6f708ab416-kube-api-access-6js2x\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:05:35.402678 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.402554    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-model-cache\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:05:35.402678 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.402563    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43fea3ff-8a26-4965-96de-db6f708ab416-tls-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:05:35.402678 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.402570    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43fea3ff-8a26-4965-96de-db6f708ab416-home\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:05:35.852189 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.852158    2577 generic.go:356] "Generic (PLEG): container finished" podID="43fea3ff-8a26-4965-96de-db6f708ab416" containerID="7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90" exitCode=0
Jul 30 20:05:35.852320 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.852197    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"
Jul 30 20:05:35.852320 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.852202    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx" event={"ID":"43fea3ff-8a26-4965-96de-db6f708ab416","Type":"ContainerDied","Data":"7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90"}
Jul 30 20:05:35.852320 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.852227    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx" event={"ID":"43fea3ff-8a26-4965-96de-db6f708ab416","Type":"ContainerDied","Data":"6503a73abe3190532d4bbd1a85a463850741479f3809f161d83d54385561d2c0"}
Jul 30 20:05:35.852320 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.852243    2577 scope.go:117] "RemoveContainer" containerID="7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90"
Jul 30 20:05:35.860235 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.860216    2577 scope.go:117] "RemoveContainer" containerID="7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90"
Jul 30 20:05:35.860498 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:05:35.860468    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90\": container with ID starting with 7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90 not found: ID does not exist" containerID="7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90"
Jul 30 20:05:35.860565 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.860502    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90"} err="failed to get container status \"7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90\": rpc error: code = NotFound desc = could not find container \"7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90\": container with ID starting with 7bfe47ba3e410f7ca3c1c5ca52e5ef02a319a4b4e727a00b86d8b13ad29a4a90 not found: ID does not exist"
Jul 30 20:05:35.876746 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.876721    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"]
Jul 30 20:05:35.879513 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:35.879489    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f6ff274e/e2e-lifecycle-model-f6ff274e-kserve-7f879ff947-52vxx"]
Jul 30 20:05:36.783125 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:36.783096    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43fea3ff-8a26-4965-96de-db6f708ab416" path="/var/lib/kubelet/pods/43fea3ff-8a26-4965-96de-db6f708ab416/volumes"
Jul 30 20:05:56.860629 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:56.860589    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"]
Jul 30 20:05:56.861213 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:56.860823    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy" containerID="cri-o://f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47" gracePeriod=30
Jul 30 20:05:59.963599 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:05:59.963559    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:06:00.003677 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:00.003635    2577 patch_prober.go:28] interesting pod/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 20:06:00.003829 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:00.003717    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 20:06:02.100944 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.100920    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:06:02.185048 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.184994    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vkrb\" (UniqueName: \"kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-kube-api-access-6vkrb\") pod \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") "
Jul 30 20:06:02.185048 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185026    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-certs\") pod \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") "
Jul 30 20:06:02.185252 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185054    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istiod-ca-cert\") pod \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") "
Jul 30 20:06:02.185252 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185103    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-podinfo\") pod \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") "
Jul 30 20:06:02.185252 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185122    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-token\") pod \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") "
Jul 30 20:06:02.185252 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185163    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-envoy\") pod \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") "
Jul 30 20:06:02.185252 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185191    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-socket\") pod \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") "
Jul 30 20:06:02.185252 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185224    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-credential-socket\") pod \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") "
Jul 30 20:06:02.185252 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185255    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-data\") pod \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\" (UID: \"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8\") "
Jul 30 20:06:02.185600 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185282    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" (UID: "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:06:02.185600 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185457    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:06:02.185600 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185488    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" (UID: "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:06:02.185600 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185553    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" (UID: "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:06:02.185600 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.185576    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" (UID: "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:06:02.186486 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.186458    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" (UID: "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:06:02.187305 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.187278    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-kube-api-access-6vkrb" (OuterVolumeSpecName: "kube-api-access-6vkrb") pod "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" (UID: "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8"). InnerVolumeSpecName "kube-api-access-6vkrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:06:02.187392 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.187318    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" (UID: "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 20:06:02.187392 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.187366    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" (UID: "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:06:02.187483 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.187465    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" (UID: "0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:06:02.286411 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.286391    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istiod-ca-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:06:02.286411 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.286409    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-podinfo\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:06:02.286525 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.286418    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:06:02.286525 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.286426    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-envoy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:06:02.286525 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.286434    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-workload-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:06:02.286525 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.286442    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-credential-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:06:02.286525 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.286451    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-istio-data\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:06:02.286525 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.286459    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vkrb\" (UniqueName: \"kubernetes.io/projected/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8-kube-api-access-6vkrb\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:06:02.970902 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.970869    2577 generic.go:356] "Generic (PLEG): container finished" podID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerID="f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47" exitCode=0
Jul 30 20:06:02.971029 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.970925    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"
Jul 30 20:06:02.971029 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.970927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" event={"ID":"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8","Type":"ContainerDied","Data":"f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47"}
Jul 30 20:06:02.971134 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.971095    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992" event={"ID":"0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8","Type":"ContainerDied","Data":"e103436c807e7248f1b9b12463431d3cd8978e47d459c2f68b25107ab20cfad0"}
Jul 30 20:06:02.971134 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.971130    2577 scope.go:117] "RemoveContainer" containerID="f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47"
Jul 30 20:06:02.979048 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.978805    2577 scope.go:117] "RemoveContainer" containerID="c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20"
Jul 30 20:06:02.985932 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.985914    2577 scope.go:117] "RemoveContainer" containerID="f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47"
Jul 30 20:06:02.986259 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:06:02.986238    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47\": container with ID starting with f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47 not found: ID does not exist" containerID="f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47"
Jul 30 20:06:02.986357 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.986265    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47"} err="failed to get container status \"f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47\": rpc error: code = NotFound desc = could not find container \"f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47\": container with ID starting with f16d050ba8f59e2fa76c6620f99dbc8855469bd6d3daa2d316046181fdedfc47 not found: ID does not exist"
Jul 30 20:06:02.986357 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.986281    2577 scope.go:117] "RemoveContainer" containerID="c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20"
Jul 30 20:06:02.986594 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:06:02.986533    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20\": container with ID starting with c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20 not found: ID does not exist" containerID="c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20"
Jul 30 20:06:02.986594 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.986562    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20"} err="failed to get container status \"c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20\": rpc error: code = NotFound desc = could not find container \"c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20\": container with ID starting with c5de9b7a4cf8c0bef63c2f223a9117e43e437db5954ca6e600673425b82d0b20 not found: ID does not exist"
Jul 30 20:06:02.987858 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.987835    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"]
Jul 30 20:06:02.994540 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:02.994519    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f6ff274e-openshift-default-56c8644dbd-58992"]
Jul 30 20:06:04.783312 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:04.783282    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" path="/var/lib/kubelet/pods/0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8/volumes"
Jul 30 20:06:22.577209 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:22.577173    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:06:32.567254 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:06:32.567172    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:07:12.140582 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.140546    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"]
Jul 30 20:07:12.141188 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.140798    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy"
Jul 30 20:07:12.141188 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.140808    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy"
Jul 30 20:07:12.141188 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.140819    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy"
Jul 30 20:07:12.141188 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.140824    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy"
Jul 30 20:07:12.141188 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.140833    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43fea3ff-8a26-4965-96de-db6f708ab416" containerName="main"
Jul 30 20:07:12.141188 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.140839    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="43fea3ff-8a26-4965-96de-db6f708ab416" containerName="main"
Jul 30 20:07:12.141188 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.140886    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="43fea3ff-8a26-4965-96de-db6f708ab416" containerName="main"
Jul 30 20:07:12.141188 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.140894    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy"
Jul 30 20:07:12.143519 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.143501    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.145948 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.145923    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-37abda-openshift-default-dockercfg-mjdb5\""
Jul 30 20:07:12.146121 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.145968    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 20:07:12.157728 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.157709    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"]
Jul 30 20:07:12.254850 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.254814    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vtkn\" (UniqueName: \"kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-kube-api-access-5vtkn\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.254968 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.254859    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-podinfo\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.254968 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.254881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-token\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.254968 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.254926    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-certs\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.254968 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.254948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-data\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.255132 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.254971    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-envoy\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.255132 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.254992    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istiod-ca-cert\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.255132 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.255030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-credential-socket\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.255132 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.255059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-socket\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.356333 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.356304    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-data\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.356455 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.356345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-envoy\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.356455 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.356373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istiod-ca-cert\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.356455 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.356400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-credential-socket\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.356455 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.356429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-socket\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.356667 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.356456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vtkn\" (UniqueName: \"kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-kube-api-access-5vtkn\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.356667 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.356484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-podinfo\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.356667 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.356515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-token\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.356667 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.356580    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-certs\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.356870 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.356848    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-data\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.357061 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.357043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-credential-socket\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.357211 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.357060    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-certs\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.357305 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.357121    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-socket\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.357305 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.357270    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istiod-ca-cert\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.358675 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.358655    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-envoy\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.358929 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.358910    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-podinfo\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.366400 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.366377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-token\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.366585 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.366569    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vtkn\" (UniqueName: \"kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-kube-api-access-5vtkn\") pod \"e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") " pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.453783 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.453730    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:12.580471 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.580444    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"]
Jul 30 20:07:12.582770 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:07:12.582744    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc990a0d2_eb4b_4238_a7c2_ac605f9d37f7.slice/crio-9b45a908598a12a982bda8caa9c55dc1de63fc1157a83a7d05025527cf05063f WatchSource:0}: Error finding container 9b45a908598a12a982bda8caa9c55dc1de63fc1157a83a7d05025527cf05063f: Status 404 returned error can't find the container with id 9b45a908598a12a982bda8caa9c55dc1de63fc1157a83a7d05025527cf05063f
Jul 30 20:07:12.584722 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.584693    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:07:12.584779 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.584757    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:07:12.584816 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:12.584785    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:07:13.242183 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:13.237508    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" event={"ID":"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7","Type":"ContainerStarted","Data":"018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff"}
Jul 30 20:07:13.242183 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:13.237681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" event={"ID":"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7","Type":"ContainerStarted","Data":"9b45a908598a12a982bda8caa9c55dc1de63fc1157a83a7d05025527cf05063f"}
Jul 30 20:07:13.259411 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:13.259364    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" podStartSLOduration=1.259350912 podStartE2EDuration="1.259350912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:07:13.258653178 +0000 UTC m=+2553.019408460" watchObservedRunningTime="2026-07-30 20:07:13.259350912 +0000 UTC m=+2553.020106187"
Jul 30 20:07:13.454776 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:13.454749    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:13.459141 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:13.459118    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:14.241090 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:14.241035    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:14.241829 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:14.241809    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:20.776768 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:20.776735    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:07:29.241927 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:29.241896    2577 patch_prober.go:28] interesting pod/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 30 20:07:29.242344 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:29.241946    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 30 20:07:29.299666 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:29.299637    2577 generic.go:356] "Generic (PLEG): container finished" podID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerID="018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff" exitCode=0
Jul 30 20:07:29.299803 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:29.299693    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" event={"ID":"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7","Type":"ContainerDied","Data":"018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff"}
Jul 30 20:07:29.300000 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:29.299986    2577 scope.go:117] "RemoveContainer" containerID="018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff"
Jul 30 20:07:29.300927 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:29.300898    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:07:29.301024 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:29.300980    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:07:29.301109 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:29.301027    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:07:30.309373 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:30.309335    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" event={"ID":"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7","Type":"ContainerStarted","Data":"d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca"}
Jul 30 20:07:30.455005 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:30.454960    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:30.455318 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:30.455282    2577 patch_prober.go:28] interesting pod/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 30 20:07:30.455397 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:30.455367    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 30 20:07:31.454312 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:31.454274    2577 patch_prober.go:28] interesting pod/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 30 20:07:31.454698 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:31.454337    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 30 20:07:32.454925 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:32.454888    2577 patch_prober.go:28] interesting pod/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 30 20:07:32.455352 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:32.454942    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 30 20:07:33.454492 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:33.454457    2577 patch_prober.go:28] interesting pod/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 30 20:07:33.454656 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:33.454524    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 30 20:07:34.454307 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:34.454271    2577 patch_prober.go:28] interesting pod/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 30 20:07:34.454699 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:34.454347    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 30 20:07:35.458708 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:35.458677    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:35.459057 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:35.458995    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:35.459652 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:35.459630    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:07:40.876585 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.876556    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"]
Jul 30 20:07:40.876925 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.876846    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b6e91bd-921b-4948-9f4f-ecf5bc5c67a8" containerName="istio-proxy"
Jul 30 20:07:40.878814 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.878797    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:40.887345 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.887322    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ff2f84-openshift-default-dockercfg-v8cxz\""
Jul 30 20:07:40.899660 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.899632    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"]
Jul 30 20:07:40.962760 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.962723    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-envoy\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:40.962897 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.962762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-token\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:40.962897 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.962785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istiod-ca-cert\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:40.962897 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.962865    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-data\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:40.963053 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.962915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-podinfo\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:40.963053 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.962945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4cfj\" (UniqueName: \"kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-kube-api-access-m4cfj\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:40.963053 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.962983    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-socket\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:40.963053 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.963013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-certs\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:40.963053 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:40.963050    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-credential-socket\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064151 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-podinfo\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064279 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4cfj\" (UniqueName: \"kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-kube-api-access-m4cfj\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064279 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-socket\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064279 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064224    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-certs\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064279 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-credential-socket\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064498 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064287    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-envoy\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064498 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-token\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064498 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istiod-ca-cert\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064498 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-data\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064720 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-socket\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064720 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064608    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-certs\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064907 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064885    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-credential-socket\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.064985 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.064970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-data\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.065206 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.065184    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istiod-ca-cert\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.066470 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.066447    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-envoy\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.066746 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.066728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-podinfo\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.074162 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.074138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4cfj\" (UniqueName: \"kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-kube-api-access-m4cfj\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.074303 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.074284    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-token\") pod \"e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") " pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.188050 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.187986    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:41.318939 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.318909    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"]
Jul 30 20:07:41.321712 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:07:41.321682    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d1bca36_bbcd_463d_868c_8f124dd40e6a.slice/crio-3f5c83509c0e388b193242c46b0bdab5018f7716330569805aa3b3a7635ae03e WatchSource:0}: Error finding container 3f5c83509c0e388b193242c46b0bdab5018f7716330569805aa3b3a7635ae03e: Status 404 returned error can't find the container with id 3f5c83509c0e388b193242c46b0bdab5018f7716330569805aa3b3a7635ae03e
Jul 30 20:07:41.323839 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.323803    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:07:41.323942 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.323880    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:07:41.324003 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.323951    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:07:41.352184 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:41.352158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" event={"ID":"8d1bca36-bbcd-463d-868c-8f124dd40e6a","Type":"ContainerStarted","Data":"3f5c83509c0e388b193242c46b0bdab5018f7716330569805aa3b3a7635ae03e"}
Jul 30 20:07:42.358158 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:42.358128    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" event={"ID":"8d1bca36-bbcd-463d-868c-8f124dd40e6a","Type":"ContainerStarted","Data":"db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843"}
Jul 30 20:07:42.378205 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:42.378161    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" podStartSLOduration=2.3781460340000002 podStartE2EDuration="2.378146034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:07:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:07:42.376892146 +0000 UTC m=+2582.137647405" watchObservedRunningTime="2026-07-30 20:07:42.378146034 +0000 UTC m=+2582.138901291"
Jul 30 20:07:43.188752 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:43.188718    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:43.193255 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:43.193228    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:43.361908 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:43.361878    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:43.362561 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:43.362543    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:07:51.167533 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:51.167502    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:07:59.076558 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:59.076522    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": read tcp 10.134.0.2:38792->10.134.0.33:15021: read: connection reset by peer" start-of-body=
Jul 30 20:07:59.076954 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:59.076592    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": read tcp 10.134.0.2:38792->10.134.0.33:15021: read: connection reset by peer"
Jul 30 20:07:59.426984 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:59.426929    2577 generic.go:356] "Generic (PLEG): container finished" podID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerID="db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843" exitCode=0
Jul 30 20:07:59.427098 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:59.426997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" event={"ID":"8d1bca36-bbcd-463d-868c-8f124dd40e6a","Type":"ContainerDied","Data":"db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843"}
Jul 30 20:07:59.427333 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:59.427318    2577 scope.go:117] "RemoveContainer" containerID="db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843"
Jul 30 20:07:59.428245 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:59.428222    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:07:59.428303 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:59.428286    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:07:59.428338 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:07:59.428318    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:08:00.433559 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:00.433514    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" event={"ID":"8d1bca36-bbcd-463d-868c-8f124dd40e6a","Type":"ContainerStarted","Data":"4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca"}
Jul 30 20:08:01.188374 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:01.188338    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:08:01.188554 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:01.188537    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 30 20:08:01.188601 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:01.188578    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 30 20:08:02.188949 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:02.188915    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 30 20:08:02.189365 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:02.188964    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 30 20:08:03.189159 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:03.189119    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 30 20:08:03.189544 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:03.189180    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 30 20:08:04.189158 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:04.189115    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Jul 30 20:08:04.189334 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:04.189171    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Jul 30 20:08:05.192235 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:05.192207    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:08:05.192625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:05.192602    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:08:05.193165 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:05.193143    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:08:13.469341 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.469310    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"]
Jul 30 20:08:13.471478 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.471462    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.474818 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.474790    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-37abda\"/\"default-dockercfg-c7dfp\""
Jul 30 20:08:13.474818 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.474820    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-37abda\"/\"test-model-37abda-kserve-self-signed-certs\""
Jul 30 20:08:13.475000 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.474907    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-37abda\"/\"openshift-service-ca.crt\""
Jul 30 20:08:13.475057 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.475040    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-37abda\"/\"kube-root-ca.crt\""
Jul 30 20:08:13.480689 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.480668    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"]
Jul 30 20:08:13.505925 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.505898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-model-cache\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.506012 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.505935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-home\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.506012 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.505969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpxz6\" (UniqueName: \"kubernetes.io/projected/49086409-5c57-4abc-ba7a-7766960b28d5-kube-api-access-wpxz6\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.506012 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.506001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-tmp-dir\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.506154 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.506038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-dshm\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.506154 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.506054    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49086409-5c57-4abc-ba7a-7766960b28d5-tls-certs\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.606342 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.606318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-dshm\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.606426 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.606347    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49086409-5c57-4abc-ba7a-7766960b28d5-tls-certs\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.606426 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.606375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-model-cache\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.606426 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.606404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-home\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.606561 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.606444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpxz6\" (UniqueName: \"kubernetes.io/projected/49086409-5c57-4abc-ba7a-7766960b28d5-kube-api-access-wpxz6\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.606608 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.606561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-tmp-dir\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.606733 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.606714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-model-cache\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.606853 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.606834    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-home\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.606916 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.606893    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-tmp-dir\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.608616 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.608598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-dshm\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.608754 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.608736    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49086409-5c57-4abc-ba7a-7766960b28d5-tls-certs\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.614347 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.614326    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpxz6\" (UniqueName: \"kubernetes.io/projected/49086409-5c57-4abc-ba7a-7766960b28d5-kube-api-access-wpxz6\") pod \"test-model-37abda-kserve-6bc58f84cc-xlflb\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") " pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.780882 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.780813    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:13.902988 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:13.902954    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"]
Jul 30 20:08:13.905766 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:08:13.905736    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49086409_5c57_4abc_ba7a_7766960b28d5.slice/crio-c83a0081c7961ed91bc20f03834f34bc54d3e1f4ebcecf6f86cd8d2789a20bab WatchSource:0}: Error finding container c83a0081c7961ed91bc20f03834f34bc54d3e1f4ebcecf6f86cd8d2789a20bab: Status 404 returned error can't find the container with id c83a0081c7961ed91bc20f03834f34bc54d3e1f4ebcecf6f86cd8d2789a20bab
Jul 30 20:08:14.496503 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:14.496467    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb" event={"ID":"49086409-5c57-4abc-ba7a-7766960b28d5","Type":"ContainerStarted","Data":"10c4dc3e37818ba88e1ba7babc3b3b79a377c8300ead457c0dc875bb15925a2f"}
Jul 30 20:08:14.496503 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:14.496506    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb" event={"ID":"49086409-5c57-4abc-ba7a-7766960b28d5","Type":"ContainerStarted","Data":"c83a0081c7961ed91bc20f03834f34bc54d3e1f4ebcecf6f86cd8d2789a20bab"}
Jul 30 20:08:14.513492 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:14.513444    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb" podStartSLOduration=1.513429425 podStartE2EDuration="1.513429425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:08:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:08:14.512504452 +0000 UTC m=+2614.273259710" watchObservedRunningTime="2026-07-30 20:08:14.513429425 +0000 UTC m=+2614.274184683"
Jul 30 20:08:18.062054 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:18.062007    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:08:23.781804 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:23.781763    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:23.781804 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:23.781813    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:23.793855 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:23.793834    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:24.552900 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:24.552875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:08:38.574148 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.574111    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"]
Jul 30 20:08:38.577644 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.577626    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.581170 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.581142    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ff2f84\"/\"kube-root-ca.crt\""
Jul 30 20:08:38.581170 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.581158    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ff2f84\"/\"default-dockercfg-5w227\""
Jul 30 20:08:38.581351 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.581156    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ff2f84\"/\"test-model-ff2f84-kserve-self-signed-certs\""
Jul 30 20:08:38.581351 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.581159    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ff2f84\"/\"openshift-service-ca.crt\""
Jul 30 20:08:38.586117 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.586095    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"]
Jul 30 20:08:38.688276 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.688252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tmp-dir\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.688390 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.688293    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chxb5\" (UniqueName: \"kubernetes.io/projected/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-kube-api-access-chxb5\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.688390 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.688321    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-model-cache\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.688390 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.688378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tls-certs\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.688511 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.688437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-home\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.688511 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.688453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-dshm\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.789829 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.789795    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tls-certs\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.790010 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.789880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-home\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.790010 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.789904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-dshm\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.790010 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.789944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tmp-dir\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.790010 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.789982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chxb5\" (UniqueName: \"kubernetes.io/projected/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-kube-api-access-chxb5\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.790010 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.790006    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-model-cache\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.790361 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.790337    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-home\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.790430 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.790399    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tmp-dir\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.790521 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.790504    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-model-cache\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.792291 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.792265    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-dshm\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.792441 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.792426    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tls-certs\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.799436 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.799415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chxb5\" (UniqueName: \"kubernetes.io/projected/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-kube-api-access-chxb5\") pod \"test-model-ff2f84-kserve-69bb485c97-q9n55\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") " pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:38.889929 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:38.889871    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:39.012041 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:39.012016    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"]
Jul 30 20:08:39.014517 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:08:39.014490    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d64b8d3_0a3d_4f99_a096_171208a0bf9c.slice/crio-76534f467353d93a6c0b9b4c3f0f2d220605e64e34767b401d311829bc6227aa WatchSource:0}: Error finding container 76534f467353d93a6c0b9b4c3f0f2d220605e64e34767b401d311829bc6227aa: Status 404 returned error can't find the container with id 76534f467353d93a6c0b9b4c3f0f2d220605e64e34767b401d311829bc6227aa
Jul 30 20:08:39.601147 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:39.601109    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55" event={"ID":"8d64b8d3-0a3d-4f99-a096-171208a0bf9c","Type":"ContainerStarted","Data":"59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12"}
Jul 30 20:08:39.601558 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:39.601155    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55" event={"ID":"8d64b8d3-0a3d-4f99-a096-171208a0bf9c","Type":"ContainerStarted","Data":"76534f467353d93a6c0b9b4c3f0f2d220605e64e34767b401d311829bc6227aa"}
Jul 30 20:08:39.618533 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:39.618486    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55" podStartSLOduration=1.6184715440000002 podStartE2EDuration="1.618471544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:08:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:08:39.618019381 +0000 UTC m=+2639.378774632" watchObservedRunningTime="2026-07-30 20:08:39.618471544 +0000 UTC m=+2639.379226836"
Jul 30 20:08:42.762781 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:42.762749    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:08:48.890280 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:48.890247    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:48.890785 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:48.890291    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:48.902460 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:48.902433    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:49.654691 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:49.654652    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:08:56.564439 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:08:56.564403    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:09:18.640432 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:18.640396    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"]
Jul 30 20:09:19.551690 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:19.551656    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"]
Jul 30 20:09:23.368277 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:23.368243    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:09:33.659063 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.658966    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb" podUID="49086409-5c57-4abc-ba7a-7766960b28d5" containerName="main" containerID="cri-o://10c4dc3e37818ba88e1ba7babc3b3b79a377c8300ead457c0dc875bb15925a2f" gracePeriod=45
Jul 30 20:09:33.833690 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.833502    2577 generic.go:356] "Generic (PLEG): container finished" podID="49086409-5c57-4abc-ba7a-7766960b28d5" containerID="10c4dc3e37818ba88e1ba7babc3b3b79a377c8300ead457c0dc875bb15925a2f" exitCode=0
Jul 30 20:09:33.833690 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.833615    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb" event={"ID":"49086409-5c57-4abc-ba7a-7766960b28d5","Type":"ContainerDied","Data":"10c4dc3e37818ba88e1ba7babc3b3b79a377c8300ead457c0dc875bb15925a2f"}
Jul 30 20:09:33.895947 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.895923    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:09:33.995982 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.995913    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpxz6\" (UniqueName: \"kubernetes.io/projected/49086409-5c57-4abc-ba7a-7766960b28d5-kube-api-access-wpxz6\") pod \"49086409-5c57-4abc-ba7a-7766960b28d5\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") "
Jul 30 20:09:33.996127 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.995983    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49086409-5c57-4abc-ba7a-7766960b28d5-tls-certs\") pod \"49086409-5c57-4abc-ba7a-7766960b28d5\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") "
Jul 30 20:09:33.996127 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.996019    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-dshm\") pod \"49086409-5c57-4abc-ba7a-7766960b28d5\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") "
Jul 30 20:09:33.996127 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.996066    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-model-cache\") pod \"49086409-5c57-4abc-ba7a-7766960b28d5\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") "
Jul 30 20:09:33.996127 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.996112    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-tmp-dir\") pod \"49086409-5c57-4abc-ba7a-7766960b28d5\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") "
Jul 30 20:09:33.996339 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.996169    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-home\") pod \"49086409-5c57-4abc-ba7a-7766960b28d5\" (UID: \"49086409-5c57-4abc-ba7a-7766960b28d5\") "
Jul 30 20:09:33.996392 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.996365    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-model-cache" (OuterVolumeSpecName: "model-cache") pod "49086409-5c57-4abc-ba7a-7766960b28d5" (UID: "49086409-5c57-4abc-ba7a-7766960b28d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:33.996450 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.996380    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "49086409-5c57-4abc-ba7a-7766960b28d5" (UID: "49086409-5c57-4abc-ba7a-7766960b28d5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:33.996492 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.996453    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-home" (OuterVolumeSpecName: "home") pod "49086409-5c57-4abc-ba7a-7766960b28d5" (UID: "49086409-5c57-4abc-ba7a-7766960b28d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:33.998278 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.998258    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-dshm" (OuterVolumeSpecName: "dshm") pod "49086409-5c57-4abc-ba7a-7766960b28d5" (UID: "49086409-5c57-4abc-ba7a-7766960b28d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:33.998278 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.998262    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49086409-5c57-4abc-ba7a-7766960b28d5-kube-api-access-wpxz6" (OuterVolumeSpecName: "kube-api-access-wpxz6") pod "49086409-5c57-4abc-ba7a-7766960b28d5" (UID: "49086409-5c57-4abc-ba7a-7766960b28d5"). InnerVolumeSpecName "kube-api-access-wpxz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:09:33.998437 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:33.998299    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49086409-5c57-4abc-ba7a-7766960b28d5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "49086409-5c57-4abc-ba7a-7766960b28d5" (UID: "49086409-5c57-4abc-ba7a-7766960b28d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:09:34.097522 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.097500    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-home\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:34.097522 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.097522    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpxz6\" (UniqueName: \"kubernetes.io/projected/49086409-5c57-4abc-ba7a-7766960b28d5-kube-api-access-wpxz6\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:34.097772 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.097532    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49086409-5c57-4abc-ba7a-7766960b28d5-tls-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:34.097772 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.097541    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-dshm\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:34.097772 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.097549    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-model-cache\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:34.097772 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.097582    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49086409-5c57-4abc-ba7a-7766960b28d5-tmp-dir\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:34.570455 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.570411    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55" podUID="8d64b8d3-0a3d-4f99-a096-171208a0bf9c" containerName="main" containerID="cri-o://59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12" gracePeriod=45
Jul 30 20:09:34.820144 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.820119    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:09:34.840385 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.840320    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"
Jul 30 20:09:34.840385 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.840342    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb" event={"ID":"49086409-5c57-4abc-ba7a-7766960b28d5","Type":"ContainerDied","Data":"c83a0081c7961ed91bc20f03834f34bc54d3e1f4ebcecf6f86cd8d2789a20bab"}
Jul 30 20:09:34.840552 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.840396    2577 scope.go:117] "RemoveContainer" containerID="10c4dc3e37818ba88e1ba7babc3b3b79a377c8300ead457c0dc875bb15925a2f"
Jul 30 20:09:34.841892 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.841839    2577 generic.go:356] "Generic (PLEG): container finished" podID="8d64b8d3-0a3d-4f99-a096-171208a0bf9c" containerID="59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12" exitCode=0
Jul 30 20:09:34.842004 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.841880    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55" event={"ID":"8d64b8d3-0a3d-4f99-a096-171208a0bf9c","Type":"ContainerDied","Data":"59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12"}
Jul 30 20:09:34.842004 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.841933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55" event={"ID":"8d64b8d3-0a3d-4f99-a096-171208a0bf9c","Type":"ContainerDied","Data":"76534f467353d93a6c0b9b4c3f0f2d220605e64e34767b401d311829bc6227aa"}
Jul 30 20:09:34.842004 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.841899    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"
Jul 30 20:09:34.849143 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.849124    2577 scope.go:117] "RemoveContainer" containerID="59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12"
Jul 30 20:09:34.855271 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.855250    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"]
Jul 30 20:09:34.857050 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.857029    2577 scope.go:117] "RemoveContainer" containerID="59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12"
Jul 30 20:09:34.857315 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:09:34.857293    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12\": container with ID starting with 59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12 not found: ID does not exist" containerID="59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12"
Jul 30 20:09:34.857378 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.857322    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12"} err="failed to get container status \"59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12\": rpc error: code = NotFound desc = could not find container \"59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12\": container with ID starting with 59a0628dd553e7e1ff987ecce71dff9f19f58a13e7e910164f3ca4ace5be5a12 not found: ID does not exist"
Jul 30 20:09:34.857485 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:34.857468    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-37abda/test-model-37abda-kserve-6bc58f84cc-xlflb"]
Jul 30 20:09:35.003930 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.003907    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-model-cache\") pod \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") "
Jul 30 20:09:35.004027 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.003938    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tmp-dir\") pod \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") "
Jul 30 20:09:35.004027 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.004006    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tls-certs\") pod \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") "
Jul 30 20:09:35.004131 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.004032    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-dshm\") pod \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") "
Jul 30 20:09:35.004131 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.004096    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chxb5\" (UniqueName: \"kubernetes.io/projected/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-kube-api-access-chxb5\") pod \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") "
Jul 30 20:09:35.004131 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.004121    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-home\") pod \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\" (UID: \"8d64b8d3-0a3d-4f99-a096-171208a0bf9c\") "
Jul 30 20:09:35.004247 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.004219    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8d64b8d3-0a3d-4f99-a096-171208a0bf9c" (UID: "8d64b8d3-0a3d-4f99-a096-171208a0bf9c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:35.004247 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.004229    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-model-cache" (OuterVolumeSpecName: "model-cache") pod "8d64b8d3-0a3d-4f99-a096-171208a0bf9c" (UID: "8d64b8d3-0a3d-4f99-a096-171208a0bf9c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:35.004435 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.004409    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-model-cache\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:35.004559 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.004443    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tmp-dir\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:35.004559 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.004415    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-home" (OuterVolumeSpecName: "home") pod "8d64b8d3-0a3d-4f99-a096-171208a0bf9c" (UID: "8d64b8d3-0a3d-4f99-a096-171208a0bf9c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:35.006158 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.006134    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8d64b8d3-0a3d-4f99-a096-171208a0bf9c" (UID: "8d64b8d3-0a3d-4f99-a096-171208a0bf9c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:09:35.006158 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.006147    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-dshm" (OuterVolumeSpecName: "dshm") pod "8d64b8d3-0a3d-4f99-a096-171208a0bf9c" (UID: "8d64b8d3-0a3d-4f99-a096-171208a0bf9c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:35.006276 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.006183    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-kube-api-access-chxb5" (OuterVolumeSpecName: "kube-api-access-chxb5") pod "8d64b8d3-0a3d-4f99-a096-171208a0bf9c" (UID: "8d64b8d3-0a3d-4f99-a096-171208a0bf9c"). InnerVolumeSpecName "kube-api-access-chxb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:09:35.105477 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.105426    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-tls-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:35.105477 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.105446    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-dshm\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:35.105477 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.105455    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chxb5\" (UniqueName: \"kubernetes.io/projected/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-kube-api-access-chxb5\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:35.105477 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.105463    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d64b8d3-0a3d-4f99-a096-171208a0bf9c-home\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:35.161552 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.161532    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"]
Jul 30 20:09:35.169261 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.169240    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-ff2f84/test-model-ff2f84-kserve-69bb485c97-q9n55"]
Jul 30 20:09:35.765902 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:35.765879    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:09:36.784151 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:36.784122    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49086409-5c57-4abc-ba7a-7766960b28d5" path="/var/lib/kubelet/pods/49086409-5c57-4abc-ba7a-7766960b28d5/volumes"
Jul 30 20:09:36.784512 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:36.784485    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d64b8d3-0a3d-4f99-a096-171208a0bf9c" path="/var/lib/kubelet/pods/8d64b8d3-0a3d-4f99-a096-171208a0bf9c/volumes"
Jul 30 20:09:36.836994 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:36.836964    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"]
Jul 30 20:09:36.837228 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:36.837200    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy" containerID="cri-o://d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca" gracePeriod=30
Jul 30 20:09:42.082090 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.082055    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:09:42.155063 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155002    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-certs\") pod \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") "
Jul 30 20:09:42.155063 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155037    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istiod-ca-cert\") pod \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") "
Jul 30 20:09:42.155063 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155061    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-token\") pod \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") "
Jul 30 20:09:42.155304 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155110    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-envoy\") pod \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") "
Jul 30 20:09:42.155304 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155145    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-data\") pod \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") "
Jul 30 20:09:42.155304 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155176    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-podinfo\") pod \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") "
Jul 30 20:09:42.155304 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155207    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-credential-socket\") pod \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") "
Jul 30 20:09:42.155304 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155242    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vtkn\" (UniqueName: \"kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-kube-api-access-5vtkn\") pod \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") "
Jul 30 20:09:42.155556 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155308    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-socket\") pod \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\" (UID: \"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7\") "
Jul 30 20:09:42.155556 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155365    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" (UID: "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:42.155556 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155501    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" (UID: "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:09:42.155556 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155525    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" (UID: "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:42.155767 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155569    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" (UID: "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:42.155767 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155617    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:42.155767 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155637    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istiod-ca-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:42.155767 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.155650    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-credential-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:42.156174 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.156144    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-data" (OuterVolumeSpecName: "istio-data") pod "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" (UID: "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:42.157461 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.157440    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" (UID: "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:42.157565 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.157499    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-kube-api-access-5vtkn" (OuterVolumeSpecName: "kube-api-access-5vtkn") pod "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" (UID: "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7"). InnerVolumeSpecName "kube-api-access-5vtkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:09:42.157894 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.157872    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" (UID: "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 20:09:42.157989 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.157891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-token" (OuterVolumeSpecName: "istio-token") pod "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" (UID: "c990a0d2-eb4b-4238-a7c2-ac605f9d37f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:09:42.256712 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.256684    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-workload-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:42.256712 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.256711    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:42.256866 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.256725    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-envoy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:42.256866 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.256737    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-data\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:42.256866 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.256748    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-istio-podinfo\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:42.256866 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.256760    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vtkn\" (UniqueName: \"kubernetes.io/projected/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7-kube-api-access-5vtkn\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:42.882767 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.882739    2577 generic.go:356] "Generic (PLEG): container finished" podID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerID="d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca" exitCode=0
Jul 30 20:09:42.882881 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.882795    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"
Jul 30 20:09:42.882881 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.882822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" event={"ID":"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7","Type":"ContainerDied","Data":"d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca"}
Jul 30 20:09:42.882948 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.882906    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d" event={"ID":"c990a0d2-eb4b-4238-a7c2-ac605f9d37f7","Type":"ContainerDied","Data":"9b45a908598a12a982bda8caa9c55dc1de63fc1157a83a7d05025527cf05063f"}
Jul 30 20:09:42.882948 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.882935    2577 scope.go:117] "RemoveContainer" containerID="d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca"
Jul 30 20:09:42.891130 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.891111    2577 scope.go:117] "RemoveContainer" containerID="018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff"
Jul 30 20:09:42.898540 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.898503    2577 scope.go:117] "RemoveContainer" containerID="d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca"
Jul 30 20:09:42.898863 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:09:42.898842    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca\": container with ID starting with d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca not found: ID does not exist" containerID="d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca"
Jul 30 20:09:42.898961 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.898869    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca"} err="failed to get container status \"d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca\": rpc error: code = NotFound desc = could not find container \"d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca\": container with ID starting with d154a24d70652372d1d1a029dfa5ceb2139e8eab4a21d0926218cbd8c09b53ca not found: ID does not exist"
Jul 30 20:09:42.898961 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.898884    2577 scope.go:117] "RemoveContainer" containerID="018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff"
Jul 30 20:09:42.899185 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:09:42.899167    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff\": container with ID starting with 018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff not found: ID does not exist" containerID="018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff"
Jul 30 20:09:42.899241 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.899190    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff"} err="failed to get container status \"018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff\": rpc error: code = NotFound desc = could not find container \"018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff\": container with ID starting with 018d729fd66a82e175d003897d50baa1f6725b7fec47538dc45e331eca762dff not found: ID does not exist"
Jul 30 20:09:42.900014 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.899997    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"]
Jul 30 20:09:42.905122 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:42.905102    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-37abda-openshift-default-6c87c997cd-pgv8d"]
Jul 30 20:09:44.783217 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:44.783185    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" path="/var/lib/kubelet/pods/c990a0d2-eb4b-4238-a7c2-ac605f9d37f7/volumes"
Jul 30 20:09:50.566525 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:50.566491    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:09:54.289668 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:54.289633    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"]
Jul 30 20:09:54.290106 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:54.289938    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy" containerID="cri-o://4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca" gracePeriod=30
Jul 30 20:09:55.158412 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.158375    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"]
Jul 30 20:09:55.158729 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.158710    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy"
Jul 30 20:09:55.158811 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.158731    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy"
Jul 30 20:09:55.158811 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.158753    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d64b8d3-0a3d-4f99-a096-171208a0bf9c" containerName="main"
Jul 30 20:09:55.158811 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.158762    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d64b8d3-0a3d-4f99-a096-171208a0bf9c" containerName="main"
Jul 30 20:09:55.158811 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.158779    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49086409-5c57-4abc-ba7a-7766960b28d5" containerName="main"
Jul 30 20:09:55.158811 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.158788    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="49086409-5c57-4abc-ba7a-7766960b28d5" containerName="main"
Jul 30 20:09:55.159084 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.158854    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy"
Jul 30 20:09:55.159084 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.158869    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d64b8d3-0a3d-4f99-a096-171208a0bf9c" containerName="main"
Jul 30 20:09:55.159084 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.158878    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="49086409-5c57-4abc-ba7a-7766960b28d5" containerName="main"
Jul 30 20:09:55.161902 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.161880    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.164125 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.164103    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ec1ee8-openshift-default-dockercfg-whq5p\""
Jul 30 20:09:55.172750 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.172726    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"]
Jul 30 20:09:55.249832 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.249800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-data\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.249943 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.249843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-token\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.249943 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.249864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-socket\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.250089 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.249926    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-certs\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.250089 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.249969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-envoy\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.250089 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.250049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-podinfo\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.250239 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.250113    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c8118b-bae6-4a51-ad7e-b36c30df3497-istiod-ca-cert\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.250239 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.250144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqtdm\" (UniqueName: \"kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-kube-api-access-zqtdm\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.250239 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.250197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-credential-socket\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.350519 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.350492    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-data\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.350914 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.350539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-token\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.350914 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.350567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-socket\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.350914 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.350598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-certs\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.350914 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.350622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-envoy\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.350914 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.350672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-podinfo\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.350914 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.350706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c8118b-bae6-4a51-ad7e-b36c30df3497-istiod-ca-cert\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.350914 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.350738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqtdm\" (UniqueName: \"kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-kube-api-access-zqtdm\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.350914 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.350776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-credential-socket\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.351356 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.350949    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-socket\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.351356 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.350980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-data\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.351356 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.351305    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-certs\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.351494 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.351406    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-credential-socket\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.351645 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.351622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c8118b-bae6-4a51-ad7e-b36c30df3497-istiod-ca-cert\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.352936 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.352916    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-envoy\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.353352 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.353329    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-podinfo\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.357589 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.357568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-token\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.357701 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.357684    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqtdm\" (UniqueName: \"kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-kube-api-access-zqtdm\") pod \"e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") " pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.472458 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.472404    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:55.605426 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.605403    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"]
Jul 30 20:09:55.607624 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:09:55.607595    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12c8118b_bae6_4a51_ad7e_b36c30df3497.slice/crio-245968b6c45ec6aad09e6097e97f72864e91def7a9cfc7eb8e4ea63759801166 WatchSource:0}: Error finding container 245968b6c45ec6aad09e6097e97f72864e91def7a9cfc7eb8e4ea63759801166: Status 404 returned error can't find the container with id 245968b6c45ec6aad09e6097e97f72864e91def7a9cfc7eb8e4ea63759801166
Jul 30 20:09:55.609850 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.609818    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:09:55.609923 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.609885    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:09:55.609923 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.609915    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:09:55.944173 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.944133    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" event={"ID":"12c8118b-bae6-4a51-ad7e-b36c30df3497","Type":"ContainerStarted","Data":"b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573"}
Jul 30 20:09:55.944350 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.944228    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" event={"ID":"12c8118b-bae6-4a51-ad7e-b36c30df3497","Type":"ContainerStarted","Data":"245968b6c45ec6aad09e6097e97f72864e91def7a9cfc7eb8e4ea63759801166"}
Jul 30 20:09:55.962502 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:55.962457    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" podStartSLOduration=0.962443666 podStartE2EDuration="962.443666ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:09:55.961352263 +0000 UTC m=+2715.722107523" watchObservedRunningTime="2026-07-30 20:09:55.962443666 +0000 UTC m=+2715.723198923"
Jul 30 20:09:56.473064 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:56.473029    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:56.477650 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:56.477627    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:56.948038 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:56.948010    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:56.948956 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:56.948936    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:09:59.528876 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.528854    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:09:59.583440 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.583412    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-envoy\") pod \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") "
Jul 30 20:09:59.583567 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.583455    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-token\") pod \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") "
Jul 30 20:09:59.583567 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.583492    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-podinfo\") pod \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") "
Jul 30 20:09:59.583567 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.583527    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-data\") pod \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") "
Jul 30 20:09:59.583739 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.583576    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4cfj\" (UniqueName: \"kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-kube-api-access-m4cfj\") pod \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") "
Jul 30 20:09:59.583739 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.583614    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istiod-ca-cert\") pod \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") "
Jul 30 20:09:59.583739 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.583683    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-socket\") pod \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") "
Jul 30 20:09:59.583739 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.583713    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-certs\") pod \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") "
Jul 30 20:09:59.583739 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.583738    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-credential-socket\") pod \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\" (UID: \"8d1bca36-bbcd-463d-868c-8f124dd40e6a\") "
Jul 30 20:09:59.584268 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.584216    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8d1bca36-bbcd-463d-868c-8f124dd40e6a" (UID: "8d1bca36-bbcd-463d-868c-8f124dd40e6a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:59.584459 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.584370    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8d1bca36-bbcd-463d-868c-8f124dd40e6a" (UID: "8d1bca36-bbcd-463d-868c-8f124dd40e6a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:59.584580 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.584455    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-data" (OuterVolumeSpecName: "istio-data") pod "8d1bca36-bbcd-463d-868c-8f124dd40e6a" (UID: "8d1bca36-bbcd-463d-868c-8f124dd40e6a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:59.584850 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.584826    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8d1bca36-bbcd-463d-868c-8f124dd40e6a" (UID: "8d1bca36-bbcd-463d-868c-8f124dd40e6a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:59.585004 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.584977    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8d1bca36-bbcd-463d-868c-8f124dd40e6a" (UID: "8d1bca36-bbcd-463d-868c-8f124dd40e6a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:09:59.585797 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.585770    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8d1bca36-bbcd-463d-868c-8f124dd40e6a" (UID: "8d1bca36-bbcd-463d-868c-8f124dd40e6a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:09:59.585885 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.585804    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-token" (OuterVolumeSpecName: "istio-token") pod "8d1bca36-bbcd-463d-868c-8f124dd40e6a" (UID: "8d1bca36-bbcd-463d-868c-8f124dd40e6a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:09:59.586097 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.586058    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-kube-api-access-m4cfj" (OuterVolumeSpecName: "kube-api-access-m4cfj") pod "8d1bca36-bbcd-463d-868c-8f124dd40e6a" (UID: "8d1bca36-bbcd-463d-868c-8f124dd40e6a"). InnerVolumeSpecName "kube-api-access-m4cfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:09:59.586097 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.586059    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8d1bca36-bbcd-463d-868c-8f124dd40e6a" (UID: "8d1bca36-bbcd-463d-868c-8f124dd40e6a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 20:09:59.684654 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.684631    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istiod-ca-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:59.684654 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.684653    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:59.684891 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.684663    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-workload-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:59.684891 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.684673    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-credential-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:59.684891 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.684686    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-envoy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:59.684891 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.684697    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:59.684891 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.684710    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-podinfo\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:59.684891 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.684722    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d1bca36-bbcd-463d-868c-8f124dd40e6a-istio-data\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:59.684891 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.684734    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4cfj\" (UniqueName: \"kubernetes.io/projected/8d1bca36-bbcd-463d-868c-8f124dd40e6a-kube-api-access-m4cfj\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:09:59.960743 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.960659    2577 generic.go:356] "Generic (PLEG): container finished" podID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerID="4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca" exitCode=0
Jul 30 20:09:59.960743 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.960707    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"
Jul 30 20:09:59.960910 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.960749    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" event={"ID":"8d1bca36-bbcd-463d-868c-8f124dd40e6a","Type":"ContainerDied","Data":"4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca"}
Jul 30 20:09:59.960910 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.960805    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt" event={"ID":"8d1bca36-bbcd-463d-868c-8f124dd40e6a","Type":"ContainerDied","Data":"3f5c83509c0e388b193242c46b0bdab5018f7716330569805aa3b3a7635ae03e"}
Jul 30 20:09:59.960910 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.960829    2577 scope.go:117] "RemoveContainer" containerID="4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca"
Jul 30 20:09:59.975238 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.975216    2577 scope.go:117] "RemoveContainer" containerID="db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843"
Jul 30 20:09:59.982779 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.982762    2577 scope.go:117] "RemoveContainer" containerID="4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca"
Jul 30 20:09:59.983097 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:09:59.983044    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca\": container with ID starting with 4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca not found: ID does not exist" containerID="4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca"
Jul 30 20:09:59.983198 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.983094    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca"} err="failed to get container status \"4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca\": rpc error: code = NotFound desc = could not find container \"4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca\": container with ID starting with 4b4bd18d2d02bf859a86b5a7b3288f6e4de638d5afe8633c8e34b52835825bca not found: ID does not exist"
Jul 30 20:09:59.983198 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.983118    2577 scope.go:117] "RemoveContainer" containerID="db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843"
Jul 30 20:09:59.983601 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:09:59.983576    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843\": container with ID starting with db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843 not found: ID does not exist" containerID="db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843"
Jul 30 20:09:59.983671 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.983609    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843"} err="failed to get container status \"db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843\": rpc error: code = NotFound desc = could not find container \"db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843\": container with ID starting with db0e86c3ff260715b02d62d9a9f4261f34ca8d1dbf28cded65628f6000919843 not found: ID does not exist"
Jul 30 20:09:59.984690 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.984672    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"]
Jul 30 20:09:59.988493 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:09:59.988470    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ff2f84-openshift-default-699fd54cc6-gl8zt"]
Jul 30 20:10:00.783469 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:00.783443    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" path="/var/lib/kubelet/pods/8d1bca36-bbcd-463d-868c-8f124dd40e6a/volumes"
Jul 30 20:10:04.664746 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:04.664714    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:10:12.390395 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:12.389696    2577 patch_prober.go:28] interesting pod/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": read tcp 10.134.0.2:46260->10.134.0.36:15021: read: connection reset by peer" start-of-body=
Jul 30 20:10:12.390395 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:12.389764    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": read tcp 10.134.0.2:46260->10.134.0.36:15021: read: connection reset by peer"
Jul 30 20:10:13.013840 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:13.013811    2577 generic.go:356] "Generic (PLEG): container finished" podID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerID="b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573" exitCode=0
Jul 30 20:10:13.013980 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:13.013865    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" event={"ID":"12c8118b-bae6-4a51-ad7e-b36c30df3497","Type":"ContainerDied","Data":"b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573"}
Jul 30 20:10:13.014196 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:13.014180    2577 scope.go:117] "RemoveContainer" containerID="b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573"
Jul 30 20:10:13.015230 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:13.015200    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:10:13.015296 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:13.015282    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:10:13.015347 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:13.015327    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:10:14.019160 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:14.019121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" event={"ID":"12c8118b-bae6-4a51-ad7e-b36c30df3497","Type":"ContainerStarted","Data":"2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349"}
Jul 30 20:10:14.472772 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:14.472731    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:10:14.472959 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:14.472936    2577 patch_prober.go:28] interesting pod/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 30 20:10:14.473028 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:14.472984    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 30 20:10:15.473341 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:15.473305    2577 patch_prober.go:28] interesting pod/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 30 20:10:15.473731 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:15.473357    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 30 20:10:16.473474 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:16.473438    2577 patch_prober.go:28] interesting pod/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 30 20:10:16.473866 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:16.473497    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 30 20:10:17.473277 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:17.473239    2577 patch_prober.go:28] interesting pod/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Jul 30 20:10:17.473454 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:17.473296    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Jul 30 20:10:18.476219 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:18.476188    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:10:18.476600 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:18.476441    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:10:18.477126 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:18.477108    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:10:23.651167 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.651134    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"]
Jul 30 20:10:23.651575 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.651466    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy"
Jul 30 20:10:23.651575 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.651483    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy"
Jul 30 20:10:23.651575 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.651510    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy"
Jul 30 20:10:23.651575 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.651518    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy"
Jul 30 20:10:23.651712 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.651593    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c990a0d2-eb4b-4238-a7c2-ac605f9d37f7" containerName="istio-proxy"
Jul 30 20:10:23.651712 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.651608    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy"
Jul 30 20:10:23.651712 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.651617    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy"
Jul 30 20:10:23.654503 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.654485    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.656604 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.656582    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-18459c-openshift-default-dockercfg-f6tc2\""
Jul 30 20:10:23.665057 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.665033    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"]
Jul 30 20:10:23.776702 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.776678    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istiod-ca-cert\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.776832 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.776716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-envoy\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.776832 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.776735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-podinfo\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.776832 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.776751    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-socket\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.776941 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.776846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hzxn\" (UniqueName: \"kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-kube-api-access-9hzxn\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.776941 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.776911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-certs\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.777006 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.776951    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-credential-socket\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.777039 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.777017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-data\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.777102 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.777044    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-token\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.877704 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.877678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-envoy\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.877810 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.877712    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-podinfo\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.877810 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.877738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-socket\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.877810 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.877769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hzxn\" (UniqueName: \"kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-kube-api-access-9hzxn\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.877955 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.877811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-certs\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.877955 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.877846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-credential-socket\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.877955 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.877884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-data\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.877955 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.877920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-token\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.878201 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.877960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istiod-ca-cert\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.878278 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.878254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-credential-socket\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.878333 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.878301    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-data\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.878455 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.878424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-socket\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.878600 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.878580    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-certs\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.878732 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.878710    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istiod-ca-cert\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.880039 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.880023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-envoy\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.880268 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.880248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-podinfo\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.885297 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.885274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-token\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.886033 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.886012    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hzxn\" (UniqueName: \"kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-kube-api-access-9hzxn\") pod \"e2e-api-b-18459c-openshift-default-64c557cf74-8qczg\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") " pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:23.966967 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:23.966913    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:24.097959 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:24.097934    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"]
Jul 30 20:10:24.103172 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:10:24.103137    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee9eee0c_2db6_48ca_bd6b_738dd8aae9e1.slice/crio-3026cce6f3aef2b5e94e52085d91bab2e7cec19c6cfef4ded77088996b8a2d58 WatchSource:0}: Error finding container 3026cce6f3aef2b5e94e52085d91bab2e7cec19c6cfef4ded77088996b8a2d58: Status 404 returned error can't find the container with id 3026cce6f3aef2b5e94e52085d91bab2e7cec19c6cfef4ded77088996b8a2d58
Jul 30 20:10:24.105231 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:24.105199    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:10:24.105296 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:24.105265    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:10:24.105335 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:24.105294    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:10:25.064856 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:25.064822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" event={"ID":"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1","Type":"ContainerStarted","Data":"fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a"}
Jul 30 20:10:25.065288 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:25.064876    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" event={"ID":"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1","Type":"ContainerStarted","Data":"3026cce6f3aef2b5e94e52085d91bab2e7cec19c6cfef4ded77088996b8a2d58"}
Jul 30 20:10:25.085308 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:25.085260    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" podStartSLOduration=2.085245755 podStartE2EDuration="2.085245755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:10:25.083646802 +0000 UTC m=+2744.844402061" watchObservedRunningTime="2026-07-30 20:10:25.085245755 +0000 UTC m=+2744.846001013"
Jul 30 20:10:25.967903 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:25.967862    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:25.973247 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:25.973212    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:26.069908 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:26.069881    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:26.070937 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:26.070917    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:32.963633 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:32.963595    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:10:41.070668 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:41.070643    2577 patch_prober.go:28] interesting pod/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 30 20:10:41.070990 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:41.070698    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 30 20:10:41.129620 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:41.129594    2577 generic.go:356] "Generic (PLEG): container finished" podID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerID="fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a" exitCode=0
Jul 30 20:10:41.129713 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:41.129632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" event={"ID":"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1","Type":"ContainerDied","Data":"fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a"}
Jul 30 20:10:41.129960 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:41.129946    2577 scope.go:117] "RemoveContainer" containerID="fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a"
Jul 30 20:10:41.130878 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:41.130848    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:10:41.130934 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:41.130919    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:10:41.130973 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:41.130961    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:10:42.139853 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:42.139815    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" event={"ID":"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1","Type":"ContainerStarted","Data":"bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a"}
Jul 30 20:10:42.967315 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:42.967278    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:42.967510 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:42.967439    2577 patch_prober.go:28] interesting pod/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 30 20:10:42.967510 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:42.967491    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 30 20:10:43.967961 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:43.967920    2577 patch_prober.go:28] interesting pod/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 30 20:10:43.968472 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:43.967984    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 30 20:10:44.968264 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:44.968223    2577 patch_prober.go:28] interesting pod/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 30 20:10:44.968620 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:44.968280    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 30 20:10:45.970906 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:45.970876    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:45.971369 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:45.971167    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:10:45.971940 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:10:45.971920    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:11:12.661481 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:12.661398    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:11:16.718096 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:16.718040    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"]
Jul 30 20:11:16.718553 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:16.718338    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy" containerID="cri-o://2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349" gracePeriod=30
Jul 30 20:11:18.478452 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:18.478411    2577 patch_prober.go:28] interesting pod/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 20:11:18.478862 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:18.478475    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 20:11:21.958889 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:21.958865    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:11:22.005623 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.005563    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-envoy\") pod \"12c8118b-bae6-4a51-ad7e-b36c30df3497\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") "
Jul 30 20:11:22.005623 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.005614    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-certs\") pod \"12c8118b-bae6-4a51-ad7e-b36c30df3497\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") "
Jul 30 20:11:22.005786 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.005646    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c8118b-bae6-4a51-ad7e-b36c30df3497-istiod-ca-cert\") pod \"12c8118b-bae6-4a51-ad7e-b36c30df3497\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") "
Jul 30 20:11:22.005786 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.005696    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-socket\") pod \"12c8118b-bae6-4a51-ad7e-b36c30df3497\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") "
Jul 30 20:11:22.005786 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.005727    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqtdm\" (UniqueName: \"kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-kube-api-access-zqtdm\") pod \"12c8118b-bae6-4a51-ad7e-b36c30df3497\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") "
Jul 30 20:11:22.005786 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.005763    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-token\") pod \"12c8118b-bae6-4a51-ad7e-b36c30df3497\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") "
Jul 30 20:11:22.006002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.005807    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-podinfo\") pod \"12c8118b-bae6-4a51-ad7e-b36c30df3497\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") "
Jul 30 20:11:22.006002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.005835    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-data\") pod \"12c8118b-bae6-4a51-ad7e-b36c30df3497\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") "
Jul 30 20:11:22.006002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.005858    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-credential-socket\") pod \"12c8118b-bae6-4a51-ad7e-b36c30df3497\" (UID: \"12c8118b-bae6-4a51-ad7e-b36c30df3497\") "
Jul 30 20:11:22.006002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.005948    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "12c8118b-bae6-4a51-ad7e-b36c30df3497" (UID: "12c8118b-bae6-4a51-ad7e-b36c30df3497"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:11:22.006245 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.006091    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12c8118b-bae6-4a51-ad7e-b36c30df3497-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "12c8118b-bae6-4a51-ad7e-b36c30df3497" (UID: "12c8118b-bae6-4a51-ad7e-b36c30df3497"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:11:22.006245 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.006129    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:22.006245 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.006182    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "12c8118b-bae6-4a51-ad7e-b36c30df3497" (UID: "12c8118b-bae6-4a51-ad7e-b36c30df3497"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:11:22.006530 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.006506    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "12c8118b-bae6-4a51-ad7e-b36c30df3497" (UID: "12c8118b-bae6-4a51-ad7e-b36c30df3497"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:11:22.007161 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.007137    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-data" (OuterVolumeSpecName: "istio-data") pod "12c8118b-bae6-4a51-ad7e-b36c30df3497" (UID: "12c8118b-bae6-4a51-ad7e-b36c30df3497"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:11:22.007952 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.007924    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-token" (OuterVolumeSpecName: "istio-token") pod "12c8118b-bae6-4a51-ad7e-b36c30df3497" (UID: "12c8118b-bae6-4a51-ad7e-b36c30df3497"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:11:22.008012 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.007963    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "12c8118b-bae6-4a51-ad7e-b36c30df3497" (UID: "12c8118b-bae6-4a51-ad7e-b36c30df3497"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:11:22.008446 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.008419    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "12c8118b-bae6-4a51-ad7e-b36c30df3497" (UID: "12c8118b-bae6-4a51-ad7e-b36c30df3497"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 20:11:22.008515 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.008440    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-kube-api-access-zqtdm" (OuterVolumeSpecName: "kube-api-access-zqtdm") pod "12c8118b-bae6-4a51-ad7e-b36c30df3497" (UID: "12c8118b-bae6-4a51-ad7e-b36c30df3497"). InnerVolumeSpecName "kube-api-access-zqtdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:11:22.106753 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.106729    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12c8118b-bae6-4a51-ad7e-b36c30df3497-istiod-ca-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:22.106753 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.106749    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-workload-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:22.106879 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.106760    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqtdm\" (UniqueName: \"kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-kube-api-access-zqtdm\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:22.106879 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.106770    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:22.106879 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.106778    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-podinfo\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:22.106879 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.106787    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-data\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:22.106879 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.106796    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-credential-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:22.106879 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.106803    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12c8118b-bae6-4a51-ad7e-b36c30df3497-istio-envoy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:22.300598 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.300575    2577 generic.go:356] "Generic (PLEG): container finished" podID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerID="2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349" exitCode=0
Jul 30 20:11:22.300709 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.300617    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"
Jul 30 20:11:22.300709 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.300641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" event={"ID":"12c8118b-bae6-4a51-ad7e-b36c30df3497","Type":"ContainerDied","Data":"2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349"}
Jul 30 20:11:22.300709 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.300686    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd" event={"ID":"12c8118b-bae6-4a51-ad7e-b36c30df3497","Type":"ContainerDied","Data":"245968b6c45ec6aad09e6097e97f72864e91def7a9cfc7eb8e4ea63759801166"}
Jul 30 20:11:22.300709 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.300708    2577 scope.go:117] "RemoveContainer" containerID="2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349"
Jul 30 20:11:22.310008 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.309994    2577 scope.go:117] "RemoveContainer" containerID="b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573"
Jul 30 20:11:22.317439 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.317417    2577 scope.go:117] "RemoveContainer" containerID="2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349"
Jul 30 20:11:22.317665 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:11:22.317646    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349\": container with ID starting with 2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349 not found: ID does not exist" containerID="2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349"
Jul 30 20:11:22.317711 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.317672    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349"} err="failed to get container status \"2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349\": rpc error: code = NotFound desc = could not find container \"2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349\": container with ID starting with 2c08c61e3a489c1b6952f456cda4f0c91d09f3c942784125373f1653e0381349 not found: ID does not exist"
Jul 30 20:11:22.317711 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.317686    2577 scope.go:117] "RemoveContainer" containerID="b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573"
Jul 30 20:11:22.317891 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:11:22.317873    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573\": container with ID starting with b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573 not found: ID does not exist" containerID="b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573"
Jul 30 20:11:22.317930 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.317898    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573"} err="failed to get container status \"b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573\": rpc error: code = NotFound desc = could not find container \"b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573\": container with ID starting with b60a303a17cc5cdfbccaa18a41ec7e936d6dd8b1ad4886b7126c8def11700573 not found: ID does not exist"
Jul 30 20:11:22.323201 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.323181    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"]
Jul 30 20:11:22.326751 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.326733    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ec1ee8-openshift-default-7fcb8774bb-84zwd"]
Jul 30 20:11:22.783153 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:22.783062    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" path="/var/lib/kubelet/pods/12c8118b-bae6-4a51-ad7e-b36c30df3497/volumes"
Jul 30 20:11:37.265336 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:37.265303    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:11:39.346732 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:39.346691    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"]
Jul 30 20:11:39.347220 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:39.346983    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerName="istio-proxy" containerID="cri-o://bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a" gracePeriod=30
Jul 30 20:11:44.594927 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.594904    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:11:44.662545 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.662474    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-certs\") pod \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") "
Jul 30 20:11:44.662545 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.662529    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-credential-socket\") pod \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") "
Jul 30 20:11:44.662722 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.662546    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-data\") pod \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") "
Jul 30 20:11:44.662722 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.662594    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istiod-ca-cert\") pod \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") "
Jul 30 20:11:44.662722 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.662612    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hzxn\" (UniqueName: \"kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-kube-api-access-9hzxn\") pod \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") "
Jul 30 20:11:44.662722 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.662631    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-socket\") pod \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") "
Jul 30 20:11:44.662722 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.662714    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-podinfo\") pod \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") "
Jul 30 20:11:44.662956 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.662743    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" (UID: "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:11:44.662956 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.662836    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" (UID: "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:11:44.663034 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.662951    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" (UID: "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:11:44.663088 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.663019    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" (UID: "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:11:44.663302 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.663280    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-token\") pod \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") "
Jul 30 20:11:44.663420 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.663321    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-envoy\") pod \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\" (UID: \"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1\") "
Jul 30 20:11:44.663538 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.663514    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" (UID: "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:11:44.663613 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.663540    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istiod-ca-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:44.663613 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.663556    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:44.663613 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.663570    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-workload-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:44.663613 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.663584    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-credential-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:44.664893 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.664854    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" (UID: "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 20:11:44.665084 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.665047    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-kube-api-access-9hzxn" (OuterVolumeSpecName: "kube-api-access-9hzxn") pod "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" (UID: "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1"). InnerVolumeSpecName "kube-api-access-9hzxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:11:44.665318 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.665288    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" (UID: "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:11:44.665642 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.665625    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" (UID: "ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:11:44.764191 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.764166    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hzxn\" (UniqueName: \"kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-kube-api-access-9hzxn\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:44.764285 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.764196    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-podinfo\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:44.764285 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.764210    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:44.764285 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.764223    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-envoy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:44.764285 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:44.764235    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1-istio-data\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:11:45.397086 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.397044    2577 generic.go:356] "Generic (PLEG): container finished" podID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerID="bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a" exitCode=0
Jul 30 20:11:45.397311 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.397133    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"
Jul 30 20:11:45.397311 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.397132    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" event={"ID":"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1","Type":"ContainerDied","Data":"bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a"}
Jul 30 20:11:45.397311 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.397231    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg" event={"ID":"ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1","Type":"ContainerDied","Data":"3026cce6f3aef2b5e94e52085d91bab2e7cec19c6cfef4ded77088996b8a2d58"}
Jul 30 20:11:45.397311 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.397264    2577 scope.go:117] "RemoveContainer" containerID="bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a"
Jul 30 20:11:45.405816 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.405797    2577 scope.go:117] "RemoveContainer" containerID="fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a"
Jul 30 20:11:45.412902 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.412881    2577 scope.go:117] "RemoveContainer" containerID="bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a"
Jul 30 20:11:45.413218 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:11:45.413185    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a\": container with ID starting with bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a not found: ID does not exist" containerID="bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a"
Jul 30 20:11:45.413384 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.413303    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a"} err="failed to get container status \"bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a\": rpc error: code = NotFound desc = could not find container \"bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a\": container with ID starting with bdcefc79204af980cec93725a0c01ddb94cec619a199f9f2b7a5af182db4b81a not found: ID does not exist"
Jul 30 20:11:45.413384 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.413334    2577 scope.go:117] "RemoveContainer" containerID="fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a"
Jul 30 20:11:45.413721 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:11:45.413678    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a\": container with ID starting with fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a not found: ID does not exist" containerID="fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a"
Jul 30 20:11:45.413829 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.413801    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a"} err="failed to get container status \"fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a\": rpc error: code = NotFound desc = could not find container \"fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a\": container with ID starting with fd730471f38f0b7ea61d1d39935116d2948ae620015d0cb3bdcdc4204fd4699a not found: ID does not exist"
Jul 30 20:11:45.414848 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.414829    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"]
Jul 30 20:11:45.420851 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:45.420831    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-18459c-openshift-default-64c557cf74-8qczg"]
Jul 30 20:11:46.782819 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:46.782786    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" path="/var/lib/kubelet/pods/ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1/volumes"
Jul 30 20:11:48.460059 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:11:48.460024    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:12:08.326807 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.326773    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"]
Jul 30 20:12:08.327211 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327184    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerName="istio-proxy"
Jul 30 20:12:08.327211 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327200    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerName="istio-proxy"
Jul 30 20:12:08.327293 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327225    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy"
Jul 30 20:12:08.327293 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327234    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy"
Jul 30 20:12:08.327293 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327249    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy"
Jul 30 20:12:08.327293 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327258    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d1bca36-bbcd-463d-868c-8f124dd40e6a" containerName="istio-proxy"
Jul 30 20:12:08.327293 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327267    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy"
Jul 30 20:12:08.327293 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327275    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy"
Jul 30 20:12:08.327293 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327286    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerName="istio-proxy"
Jul 30 20:12:08.327293 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327293    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerName="istio-proxy"
Jul 30 20:12:08.327558 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327360    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerName="istio-proxy"
Jul 30 20:12:08.327558 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327372    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy"
Jul 30 20:12:08.327558 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327382    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="12c8118b-bae6-4a51-ad7e-b36c30df3497" containerName="istio-proxy"
Jul 30 20:12:08.327558 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.327392    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9eee0c-2db6-48ca-bd6b-738dd8aae9e1" containerName="istio-proxy"
Jul 30 20:12:08.330450 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.330430    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.332935 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.332915    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 20:12:08.333057 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.332916    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-a769e5-openshift-default-dockercfg-8dpn9\""
Jul 30 20:12:08.343211 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.343187    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"]
Jul 30 20:12:08.412301 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.412274    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-certs\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.412385 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.412308    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8g76\" (UniqueName: \"kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-kube-api-access-t8g76\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.412385 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.412328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-credential-socket\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.412458 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.412402    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-token\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.412458 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.412427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-envoy\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.412527 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.412457    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-podinfo\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.412527 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.412516    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-socket\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.412591 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.412541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istiod-ca-cert\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.412591 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.412560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-data\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513119 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513098    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-token\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513214 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-envoy\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513214 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513146    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-podinfo\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513214 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-socket\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513214 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istiod-ca-cert\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513373 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-data\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513427 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513387    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-certs\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513427 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8g76\" (UniqueName: \"kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-kube-api-access-t8g76\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513528 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-credential-socket\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513604 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513586    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-socket\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513745 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513704    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-data\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513929 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-credential-socket\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.513929 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.513903    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-certs\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.514034 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.514004    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istiod-ca-cert\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.516105 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.516057    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-envoy\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.516268 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.516253    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-podinfo\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.523540 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.523509    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-token\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.523870 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.523846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8g76\" (UniqueName: \"kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-kube-api-access-t8g76\") pod \"e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") " pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.641411 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.641350    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:08.771404 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.771379    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"]
Jul 30 20:12:08.774450 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:12:08.774414    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef11c24_cd63_40d6_8ac7_872c3a6858dd.slice/crio-c3c57edb9f5e409816d666a8e394cf51c7e1bfd4d1d5a047b3381fb49c1afa83 WatchSource:0}: Error finding container c3c57edb9f5e409816d666a8e394cf51c7e1bfd4d1d5a047b3381fb49c1afa83: Status 404 returned error can't find the container with id c3c57edb9f5e409816d666a8e394cf51c7e1bfd4d1d5a047b3381fb49c1afa83
Jul 30 20:12:08.776752 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.776716    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:12:08.776859 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.776782    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:12:08.776859 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:08.776809    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:12:09.494844 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:09.494808    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" event={"ID":"5ef11c24-cd63-40d6-8ac7-872c3a6858dd","Type":"ContainerStarted","Data":"00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1"}
Jul 30 20:12:09.495264 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:09.494896    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" event={"ID":"5ef11c24-cd63-40d6-8ac7-872c3a6858dd","Type":"ContainerStarted","Data":"c3c57edb9f5e409816d666a8e394cf51c7e1bfd4d1d5a047b3381fb49c1afa83"}
Jul 30 20:12:09.518907 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:09.518853    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" podStartSLOduration=1.518837484 podStartE2EDuration="1.518837484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:12:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:12:09.517275017 +0000 UTC m=+2849.278030275" watchObservedRunningTime="2026-07-30 20:12:09.518837484 +0000 UTC m=+2849.279592741"
Jul 30 20:12:09.642348 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:09.642321    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:09.646507 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:09.646486    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:10.499815 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:10.499781    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:10.500678 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:10.500658    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:12.501207 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.501176    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"]
Jul 30 20:12:12.504387 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.504363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:12.506972 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.506946    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 20:12:12.507939 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.507917    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 20:12:12.508382 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.508358    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-a769e5-serving-cert\""
Jul 30 20:12:12.508581 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.508562    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6qhn5\""
Jul 30 20:12:12.516895 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.516874    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"]
Jul 30 20:12:12.644007 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.643981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qnr2\" (UniqueName: \"kubernetes.io/projected/ce5b8960-f89b-40f2-8761-3e45de27fbbb-kube-api-access-6qnr2\") pod \"maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:12.644173 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.644118    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce5b8960-f89b-40f2-8761-3e45de27fbbb-maas-api-tls\") pod \"maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:12.644173 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.644154    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce5b8960-f89b-40f2-8761-3e45de27fbbb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:12.744934 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.744913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce5b8960-f89b-40f2-8761-3e45de27fbbb-maas-api-tls\") pod \"maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:12.745058 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.744956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce5b8960-f89b-40f2-8761-3e45de27fbbb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:12.745058 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.744997    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qnr2\" (UniqueName: \"kubernetes.io/projected/ce5b8960-f89b-40f2-8761-3e45de27fbbb-kube-api-access-6qnr2\") pod \"maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:12.745165 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:12:12.745064    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-serving-cert: secret "maas-api-e2e-shared-b-a769e5-serving-cert" not found
Jul 30 20:12:12.745165 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:12:12.745152    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce5b8960-f89b-40f2-8761-3e45de27fbbb-maas-api-tls podName:ce5b8960-f89b-40f2-8761-3e45de27fbbb nodeName:}" failed. No retries permitted until 2026-07-30 20:12:13.245131733 +0000 UTC m=+2853.005886971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce5b8960-f89b-40f2-8761-3e45de27fbbb-maas-api-tls") pod "maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw" (UID: "ce5b8960-f89b-40f2-8761-3e45de27fbbb") : secret "maas-api-e2e-shared-b-a769e5-serving-cert" not found
Jul 30 20:12:12.745654 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.745635    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce5b8960-f89b-40f2-8761-3e45de27fbbb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:12.755206 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:12.755155    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qnr2\" (UniqueName: \"kubernetes.io/projected/ce5b8960-f89b-40f2-8761-3e45de27fbbb-kube-api-access-6qnr2\") pod \"maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:13.249704 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:13.249666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce5b8960-f89b-40f2-8761-3e45de27fbbb-maas-api-tls\") pod \"maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:13.252080 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:13.252050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce5b8960-f89b-40f2-8761-3e45de27fbbb-maas-api-tls\") pod \"maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:13.419855 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:13.419823    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:13.544248 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:13.544176    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"]
Jul 30 20:12:13.546934 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:12:13.546903    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce5b8960_f89b_40f2_8761_3e45de27fbbb.slice/crio-0a7e8b48ab3ed8cd37b02b243acd0459a6fbcab564037e2e7bac7be396dbdbce WatchSource:0}: Error finding container 0a7e8b48ab3ed8cd37b02b243acd0459a6fbcab564037e2e7bac7be396dbdbce: Status 404 returned error can't find the container with id 0a7e8b48ab3ed8cd37b02b243acd0459a6fbcab564037e2e7bac7be396dbdbce
Jul 30 20:12:13.548287 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:13.548266    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:12:14.520457 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:14.520408    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw" event={"ID":"ce5b8960-f89b-40f2-8761-3e45de27fbbb","Type":"ContainerStarted","Data":"0a7e8b48ab3ed8cd37b02b243acd0459a6fbcab564037e2e7bac7be396dbdbce"}
Jul 30 20:12:16.532599 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:16.532563    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw" event={"ID":"ce5b8960-f89b-40f2-8761-3e45de27fbbb","Type":"ContainerStarted","Data":"3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9"}
Jul 30 20:12:16.533055 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:16.532685    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:16.550013 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:16.549966    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw" podStartSLOduration=2.307796873 podStartE2EDuration="4.549955744s" totalImagesPullingTime="2.242158871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:12:16.548719908 +0000 UTC m=+2856.309475166" watchObservedRunningTime="2026-07-30 20:12:16.549955744 +0000 UTC m=+2856.310711001"
Jul 30 20:12:18.761505 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:18.761473    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:12:22.553337 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:22.553304    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:12:26.304658 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:26.304624    2577 patch_prober.go:28] interesting pod/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": read tcp 10.134.0.2:46136->10.134.0.38:15021: read: connection reset by peer" start-of-body=
Jul 30 20:12:26.305111 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:26.304703    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": read tcp 10.134.0.2:46136->10.134.0.38:15021: read: connection reset by peer"
Jul 30 20:12:26.578965 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:26.578897    2577 generic.go:356] "Generic (PLEG): container finished" podID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerID="00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1" exitCode=0
Jul 30 20:12:26.578965 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:26.578956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" event={"ID":"5ef11c24-cd63-40d6-8ac7-872c3a6858dd","Type":"ContainerDied","Data":"00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1"}
Jul 30 20:12:26.579289 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:26.579274    2577 scope.go:117] "RemoveContainer" containerID="00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1"
Jul 30 20:12:26.580236 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:26.580211    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:12:26.580304 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:26.580278    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:12:26.580343 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:26.580305    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:12:27.584334 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:27.584297    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" event={"ID":"5ef11c24-cd63-40d6-8ac7-872c3a6858dd","Type":"ContainerStarted","Data":"8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d"}
Jul 30 20:12:27.642486 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:27.642440    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:27.642693 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:27.642573    2577 patch_prober.go:28] interesting pod/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 30 20:12:27.642693 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:27.642621    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 30 20:12:28.642442 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:28.641977    2577 patch_prober.go:28] interesting pod/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 30 20:12:28.642442 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:28.642038    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 30 20:12:29.642196 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:29.642161    2577 patch_prober.go:28] interesting pod/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 30 20:12:29.642479 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:29.642451    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 30 20:12:30.642539 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:30.642497    2577 patch_prober.go:28] interesting pod/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 30 20:12:30.643030 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:30.642559    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 30 20:12:31.645267 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:31.645236    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:31.645656 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:31.645483    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:31.646113 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:31.646094    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:12:46.561843 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.561783    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"]
Jul 30 20:12:46.566289 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.566269    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.569775 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.569728    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ecda00\"/\"openshift-service-ca.crt\""
Jul 30 20:12:46.569775 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.569733    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ecda00\"/\"kube-root-ca.crt\""
Jul 30 20:12:46.569953 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.569806    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ecda00\"/\"auth-test-model-ecda00-kserve-self-signed-certs\""
Jul 30 20:12:46.569953 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.569759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ecda00\"/\"default-dockercfg-6vb87\""
Jul 30 20:12:46.575892 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.575870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"]
Jul 30 20:12:46.694173 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.694149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-dshm\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.694291 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.694185    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-tmp-dir\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.694291 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.694208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t9fz\" (UniqueName: \"kubernetes.io/projected/123277b5-4ffc-47c9-a54b-9294ecbbd725-kube-api-access-4t9fz\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.694416 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.694302    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-home\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.694416 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.694335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123277b5-4ffc-47c9-a54b-9294ecbbd725-tls-certs\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.694416 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.694390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-model-cache\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.795244 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.795218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-model-cache\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.795382 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.795253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-dshm\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.795382 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.795284    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-tmp-dir\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.795382 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.795306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t9fz\" (UniqueName: \"kubernetes.io/projected/123277b5-4ffc-47c9-a54b-9294ecbbd725-kube-api-access-4t9fz\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.795382 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.795339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-home\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.795382 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.795356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123277b5-4ffc-47c9-a54b-9294ecbbd725-tls-certs\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.795734 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.795714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-model-cache\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.795802 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.795756    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-home\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.795844 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.795809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-tmp-dir\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.797409 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.797391    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-dshm\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.797701 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.797684    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123277b5-4ffc-47c9-a54b-9294ecbbd725-tls-certs\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.809681 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.809654    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t9fz\" (UniqueName: \"kubernetes.io/projected/123277b5-4ffc-47c9-a54b-9294ecbbd725-kube-api-access-4t9fz\") pod \"auth-test-model-ecda00-kserve-b5b745947-kc662\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") " pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:46.875901 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:46.875846    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:47.003846 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:47.003687    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"]
Jul 30 20:12:47.005928 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:12:47.005897    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod123277b5_4ffc_47c9_a54b_9294ecbbd725.slice/crio-acba4d3cf01780b8a6267ab151a810ab8e79644433d6db26893ac3b1fd1572cf WatchSource:0}: Error finding container acba4d3cf01780b8a6267ab151a810ab8e79644433d6db26893ac3b1fd1572cf: Status 404 returned error can't find the container with id acba4d3cf01780b8a6267ab151a810ab8e79644433d6db26893ac3b1fd1572cf
Jul 30 20:12:47.662553 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:47.662519    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662" event={"ID":"123277b5-4ffc-47c9-a54b-9294ecbbd725","Type":"ContainerStarted","Data":"26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8"}
Jul 30 20:12:47.662553 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:47.662560    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662" event={"ID":"123277b5-4ffc-47c9-a54b-9294ecbbd725","Type":"ContainerStarted","Data":"acba4d3cf01780b8a6267ab151a810ab8e79644433d6db26893ac3b1fd1572cf"}
Jul 30 20:12:47.679877 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:47.679828    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662" podStartSLOduration=1.679813392 podStartE2EDuration="1.679813392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:12:47.678232819 +0000 UTC m=+2887.438988080" watchObservedRunningTime="2026-07-30 20:12:47.679813392 +0000 UTC m=+2887.440568650"
Jul 30 20:12:50.365807 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:50.365775    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:12:56.875962 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:56.875922    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:56.876365 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:56.875971    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:56.888033 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:56.888009    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:12:57.716509 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:12:57.716485    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:13:26.062731 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:13:26.062698    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:13:58.163354 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:13:58.163318    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:14:23.571842 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:14:23.571804    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:14:41.463184 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:14:41.463146    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:15:07.260852 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:15:07.260814    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:15:24.568791 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:15:24.568757    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:15:51.158485 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:15:51.158402    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:16:10.461136 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:16:10.461101    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:16:37.079343 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:16:37.079309    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:16:54.263397 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:16:54.263363    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:17:21.164259 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:17:21.164181    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:17:37.167927 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:17:37.167895    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:18:03.293032 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:03.292997    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:18:06.113650 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:06.113620    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"]
Jul 30 20:18:15.876805 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:15.876771    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"]
Jul 30 20:18:15.880147 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:15.880125    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:15.882806 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:15.882786    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ecda00\"/\"sub-test-model-ecda00-kserve-self-signed-certs\""
Jul 30 20:18:15.894395 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:15.894369    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"]
Jul 30 20:18:15.982524 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:15.982500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-dshm\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:15.982638 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:15.982542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-tmp-dir\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:15.982638 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:15.982568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-model-cache\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:15.982759 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:15.982664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-home\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:15.982759 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:15.982704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48fb8f43-4586-4552-9d06-cbc11852d887-tls-certs\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:15.982837 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:15.982757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkkgq\" (UniqueName: \"kubernetes.io/projected/48fb8f43-4586-4552-9d06-cbc11852d887-kube-api-access-lkkgq\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.083732 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.083708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-dshm\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.083835 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.083777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-tmp-dir\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.083835 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.083814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-model-cache\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.083915 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.083886    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-home\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.083955 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.083931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48fb8f43-4586-4552-9d06-cbc11852d887-tls-certs\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.083999 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.083963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkkgq\" (UniqueName: \"kubernetes.io/projected/48fb8f43-4586-4552-9d06-cbc11852d887-kube-api-access-lkkgq\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.084238 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.084214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-model-cache\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.084361 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.084249    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-home\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.084361 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.084283    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-tmp-dir\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.085942 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.085922    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-dshm\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.086431 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.086406    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48fb8f43-4586-4552-9d06-cbc11852d887-tls-certs\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.092191 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.092167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkkgq\" (UniqueName: \"kubernetes.io/projected/48fb8f43-4586-4552-9d06-cbc11852d887-kube-api-access-lkkgq\") pod \"sub-test-model-ecda00-kserve-747867bfb9-79j5m\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") " pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.188880 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.188822    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:16.317425 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.317390    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"]
Jul 30 20:18:16.320174 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:18:16.320142    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48fb8f43_4586_4552_9d06_cbc11852d887.slice/crio-29dd33dc40403e5c911c21dce79bb0470d9a8789b0bd9dca540566d2b3b763bc WatchSource:0}: Error finding container 29dd33dc40403e5c911c21dce79bb0470d9a8789b0bd9dca540566d2b3b763bc: Status 404 returned error can't find the container with id 29dd33dc40403e5c911c21dce79bb0470d9a8789b0bd9dca540566d2b3b763bc
Jul 30 20:18:16.927031 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.926994    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m" event={"ID":"48fb8f43-4586-4552-9d06-cbc11852d887","Type":"ContainerStarted","Data":"37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484"}
Jul 30 20:18:16.927031 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.927029    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m" event={"ID":"48fb8f43-4586-4552-9d06-cbc11852d887","Type":"ContainerStarted","Data":"29dd33dc40403e5c911c21dce79bb0470d9a8789b0bd9dca540566d2b3b763bc"}
Jul 30 20:18:16.946805 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:16.946759    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m" podStartSLOduration=1.946745159 podStartE2EDuration="1.946745159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:18:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:18:16.94601853 +0000 UTC m=+3216.706773789" watchObservedRunningTime="2026-07-30 20:18:16.946745159 +0000 UTC m=+3216.707500467"
Jul 30 20:18:19.564510 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:19.564474    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:18:21.131726 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.131691    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662" podUID="123277b5-4ffc-47c9-a54b-9294ecbbd725" containerName="main" containerID="cri-o://26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8" gracePeriod=45
Jul 30 20:18:21.375851 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.375827    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:18:21.417265 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.417196    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-model-cache\") pod \"123277b5-4ffc-47c9-a54b-9294ecbbd725\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") "
Jul 30 20:18:21.417265 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.417258    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-dshm\") pod \"123277b5-4ffc-47c9-a54b-9294ecbbd725\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") "
Jul 30 20:18:21.417452 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.417304    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t9fz\" (UniqueName: \"kubernetes.io/projected/123277b5-4ffc-47c9-a54b-9294ecbbd725-kube-api-access-4t9fz\") pod \"123277b5-4ffc-47c9-a54b-9294ecbbd725\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") "
Jul 30 20:18:21.417452 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.417346    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-home\") pod \"123277b5-4ffc-47c9-a54b-9294ecbbd725\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") "
Jul 30 20:18:21.417452 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.417376    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123277b5-4ffc-47c9-a54b-9294ecbbd725-tls-certs\") pod \"123277b5-4ffc-47c9-a54b-9294ecbbd725\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") "
Jul 30 20:18:21.417452 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.417430    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-tmp-dir\") pod \"123277b5-4ffc-47c9-a54b-9294ecbbd725\" (UID: \"123277b5-4ffc-47c9-a54b-9294ecbbd725\") "
Jul 30 20:18:21.417452 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.417436    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-model-cache" (OuterVolumeSpecName: "model-cache") pod "123277b5-4ffc-47c9-a54b-9294ecbbd725" (UID: "123277b5-4ffc-47c9-a54b-9294ecbbd725"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:18:21.417687 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.417590    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-home" (OuterVolumeSpecName: "home") pod "123277b5-4ffc-47c9-a54b-9294ecbbd725" (UID: "123277b5-4ffc-47c9-a54b-9294ecbbd725"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:18:21.418186 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.418162    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-model-cache\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:18:21.418303 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.418200    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-home\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:18:21.418303 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.418222    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "123277b5-4ffc-47c9-a54b-9294ecbbd725" (UID: "123277b5-4ffc-47c9-a54b-9294ecbbd725"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:18:21.420673 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.420636    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/123277b5-4ffc-47c9-a54b-9294ecbbd725-kube-api-access-4t9fz" (OuterVolumeSpecName: "kube-api-access-4t9fz") pod "123277b5-4ffc-47c9-a54b-9294ecbbd725" (UID: "123277b5-4ffc-47c9-a54b-9294ecbbd725"). InnerVolumeSpecName "kube-api-access-4t9fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:18:21.420793 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.420692    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-dshm" (OuterVolumeSpecName: "dshm") pod "123277b5-4ffc-47c9-a54b-9294ecbbd725" (UID: "123277b5-4ffc-47c9-a54b-9294ecbbd725"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:18:21.424114 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.422663    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/123277b5-4ffc-47c9-a54b-9294ecbbd725-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "123277b5-4ffc-47c9-a54b-9294ecbbd725" (UID: "123277b5-4ffc-47c9-a54b-9294ecbbd725"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:18:21.519302 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.519283    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/123277b5-4ffc-47c9-a54b-9294ecbbd725-tls-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:18:21.519302 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.519301    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-tmp-dir\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:18:21.519428 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.519310    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/123277b5-4ffc-47c9-a54b-9294ecbbd725-dshm\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:18:21.519428 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.519319    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t9fz\" (UniqueName: \"kubernetes.io/projected/123277b5-4ffc-47c9-a54b-9294ecbbd725-kube-api-access-4t9fz\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:18:21.947458 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.947432    2577 generic.go:356] "Generic (PLEG): container finished" podID="123277b5-4ffc-47c9-a54b-9294ecbbd725" containerID="26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8" exitCode=0
Jul 30 20:18:21.947592 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.947469    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662" event={"ID":"123277b5-4ffc-47c9-a54b-9294ecbbd725","Type":"ContainerDied","Data":"26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8"}
Jul 30 20:18:21.947592 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.947485    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"
Jul 30 20:18:21.947592 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.947501    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662" event={"ID":"123277b5-4ffc-47c9-a54b-9294ecbbd725","Type":"ContainerDied","Data":"acba4d3cf01780b8a6267ab151a810ab8e79644433d6db26893ac3b1fd1572cf"}
Jul 30 20:18:21.947592 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.947522    2577 scope.go:117] "RemoveContainer" containerID="26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8"
Jul 30 20:18:21.955490 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.955470    2577 scope.go:117] "RemoveContainer" containerID="26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8"
Jul 30 20:18:21.955754 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:18:21.955732    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8\": container with ID starting with 26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8 not found: ID does not exist" containerID="26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8"
Jul 30 20:18:21.955833 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.955763    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8"} err="failed to get container status \"26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8\": rpc error: code = NotFound desc = could not find container \"26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8\": container with ID starting with 26cdca3d12e40cb84f188930c8cfe70f755ec3a80b00c92a9ad6d03718e1fcc8 not found: ID does not exist"
Jul 30 20:18:21.967944 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.967923    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"]
Jul 30 20:18:21.971471 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:21.971451    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/auth-test-model-ecda00-kserve-b5b745947-kc662"]
Jul 30 20:18:22.784371 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:22.784341    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="123277b5-4ffc-47c9-a54b-9294ecbbd725" path="/var/lib/kubelet/pods/123277b5-4ffc-47c9-a54b-9294ecbbd725/volumes"
Jul 30 20:18:26.189304 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:26.189273    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:26.189755 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:26.189316    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:26.201592 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:26.201574    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:26.978862 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:26.978839    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:18:56.866786 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:18:56.866751    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:19:57.269266 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:19:57.269231    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:20:15.868119 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:15.868018    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:20:38.363904 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:38.363870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:20:43.651576 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:43.651543    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"]
Jul 30 20:20:55.371674 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.371641    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:20:55.888368 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.888341    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"]
Jul 30 20:20:55.888625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.888612    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="123277b5-4ffc-47c9-a54b-9294ecbbd725" containerName="main"
Jul 30 20:20:55.888671 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.888625    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="123277b5-4ffc-47c9-a54b-9294ecbbd725" containerName="main"
Jul 30 20:20:55.888705 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.888674    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="123277b5-4ffc-47c9-a54b-9294ecbbd725" containerName="main"
Jul 30 20:20:55.891448 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.891432    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:55.894025 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.893999    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ecda00\"/\"rate-test-model-ecda00-kserve-self-signed-certs\""
Jul 30 20:20:55.900761 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.900739    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"]
Jul 30 20:20:55.971890 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.971864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-model-cache\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:55.971890 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.971896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bc56200-a70c-4d2f-8321-06aa160876f6-tls-certs\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:55.972054 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.971914    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-tmp-dir\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:55.972054 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.971935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-dshm\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:55.972054 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.972008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-home\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:55.972174 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:55.972093    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28jqt\" (UniqueName: \"kubernetes.io/projected/6bc56200-a70c-4d2f-8321-06aa160876f6-kube-api-access-28jqt\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.072886 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.072861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-dshm\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.072977 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.072895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-home\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.072977 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.072928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28jqt\" (UniqueName: \"kubernetes.io/projected/6bc56200-a70c-4d2f-8321-06aa160876f6-kube-api-access-28jqt\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.073098 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.072987    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-model-cache\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.073098 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.073014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bc56200-a70c-4d2f-8321-06aa160876f6-tls-certs\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.073098 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.073039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-tmp-dir\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.073274 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.073254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-home\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.073369 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.073354    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-tmp-dir\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.073428 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.073408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-model-cache\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.075121 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.075096    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-dshm\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.075424 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.075405    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bc56200-a70c-4d2f-8321-06aa160876f6-tls-certs\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.081396 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.081366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28jqt\" (UniqueName: \"kubernetes.io/projected/6bc56200-a70c-4d2f-8321-06aa160876f6-kube-api-access-28jqt\") pod \"rate-test-model-ecda00-kserve-75df8b4c67-vh28h\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") " pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.201400 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.201318    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:20:56.327974 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.327945    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"]
Jul 30 20:20:56.331410 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:20:56.331378    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bc56200_a70c_4d2f_8321_06aa160876f6.slice/crio-ab9424c110ca635835c6eee879f89fbfc7e7ecf9fb9a70da1b32463a1271ecc7 WatchSource:0}: Error finding container ab9424c110ca635835c6eee879f89fbfc7e7ecf9fb9a70da1b32463a1271ecc7: Status 404 returned error can't find the container with id ab9424c110ca635835c6eee879f89fbfc7e7ecf9fb9a70da1b32463a1271ecc7
Jul 30 20:20:56.551491 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.551460    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h" event={"ID":"6bc56200-a70c-4d2f-8321-06aa160876f6","Type":"ContainerStarted","Data":"90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf"}
Jul 30 20:20:56.551843 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.551501    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h" event={"ID":"6bc56200-a70c-4d2f-8321-06aa160876f6","Type":"ContainerStarted","Data":"ab9424c110ca635835c6eee879f89fbfc7e7ecf9fb9a70da1b32463a1271ecc7"}
Jul 30 20:20:56.569962 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:56.569914    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h" podStartSLOduration=1.569900405 podStartE2EDuration="1.569900405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:20:56.568776446 +0000 UTC m=+3376.329531694" watchObservedRunningTime="2026-07-30 20:20:56.569900405 +0000 UTC m=+3376.330655665"
Jul 30 20:20:58.669344 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.669305    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m" podUID="48fb8f43-4586-4552-9d06-cbc11852d887" containerName="main" containerID="cri-o://37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484" gracePeriod=45
Jul 30 20:20:58.915161 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.915137    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:20:58.996337 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.996283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48fb8f43-4586-4552-9d06-cbc11852d887-tls-certs\") pod \"48fb8f43-4586-4552-9d06-cbc11852d887\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") "
Jul 30 20:20:58.996337 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.996320    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-tmp-dir\") pod \"48fb8f43-4586-4552-9d06-cbc11852d887\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") "
Jul 30 20:20:58.996337 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.996339    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-home\") pod \"48fb8f43-4586-4552-9d06-cbc11852d887\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") "
Jul 30 20:20:58.996583 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.996482    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-dshm\") pod \"48fb8f43-4586-4552-9d06-cbc11852d887\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") "
Jul 30 20:20:58.996583 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.996541    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkkgq\" (UniqueName: \"kubernetes.io/projected/48fb8f43-4586-4552-9d06-cbc11852d887-kube-api-access-lkkgq\") pod \"48fb8f43-4586-4552-9d06-cbc11852d887\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") "
Jul 30 20:20:58.996583 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.996573    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-home" (OuterVolumeSpecName: "home") pod "48fb8f43-4586-4552-9d06-cbc11852d887" (UID: "48fb8f43-4586-4552-9d06-cbc11852d887"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:20:58.996730 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.996583    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "48fb8f43-4586-4552-9d06-cbc11852d887" (UID: "48fb8f43-4586-4552-9d06-cbc11852d887"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:20:58.996730 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.996596    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-model-cache\") pod \"48fb8f43-4586-4552-9d06-cbc11852d887\" (UID: \"48fb8f43-4586-4552-9d06-cbc11852d887\") "
Jul 30 20:20:58.996847 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.996829    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-tmp-dir\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:20:58.996895 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.996852    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-home\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:20:58.996964 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.996886    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-model-cache" (OuterVolumeSpecName: "model-cache") pod "48fb8f43-4586-4552-9d06-cbc11852d887" (UID: "48fb8f43-4586-4552-9d06-cbc11852d887"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:20:58.998340 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.998311    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48fb8f43-4586-4552-9d06-cbc11852d887-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "48fb8f43-4586-4552-9d06-cbc11852d887" (UID: "48fb8f43-4586-4552-9d06-cbc11852d887"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:20:58.998448 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.998361    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-dshm" (OuterVolumeSpecName: "dshm") pod "48fb8f43-4586-4552-9d06-cbc11852d887" (UID: "48fb8f43-4586-4552-9d06-cbc11852d887"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:20:58.998509 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:58.998489    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48fb8f43-4586-4552-9d06-cbc11852d887-kube-api-access-lkkgq" (OuterVolumeSpecName: "kube-api-access-lkkgq") pod "48fb8f43-4586-4552-9d06-cbc11852d887" (UID: "48fb8f43-4586-4552-9d06-cbc11852d887"). InnerVolumeSpecName "kube-api-access-lkkgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:20:59.097772 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.097750    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-dshm\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:20:59.097772 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.097772    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkkgq\" (UniqueName: \"kubernetes.io/projected/48fb8f43-4586-4552-9d06-cbc11852d887-kube-api-access-lkkgq\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:20:59.097905 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.097783    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48fb8f43-4586-4552-9d06-cbc11852d887-model-cache\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:20:59.097905 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.097792    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48fb8f43-4586-4552-9d06-cbc11852d887-tls-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:20:59.564511 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.564477    2577 generic.go:356] "Generic (PLEG): container finished" podID="48fb8f43-4586-4552-9d06-cbc11852d887" containerID="37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484" exitCode=0
Jul 30 20:20:59.564650 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.564525    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"
Jul 30 20:20:59.564650 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.564529    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m" event={"ID":"48fb8f43-4586-4552-9d06-cbc11852d887","Type":"ContainerDied","Data":"37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484"}
Jul 30 20:20:59.564650 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.564567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m" event={"ID":"48fb8f43-4586-4552-9d06-cbc11852d887","Type":"ContainerDied","Data":"29dd33dc40403e5c911c21dce79bb0470d9a8789b0bd9dca540566d2b3b763bc"}
Jul 30 20:20:59.564650 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.564599    2577 scope.go:117] "RemoveContainer" containerID="37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484"
Jul 30 20:20:59.572882 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.572863    2577 scope.go:117] "RemoveContainer" containerID="37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484"
Jul 30 20:20:59.573170 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:20:59.573143    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484\": container with ID starting with 37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484 not found: ID does not exist" containerID="37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484"
Jul 30 20:20:59.573219 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.573182    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484"} err="failed to get container status \"37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484\": rpc error: code = NotFound desc = could not find container \"37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484\": container with ID starting with 37f001fbd4ac60191c5bb1d08538d61862e919bb8eae4f0a922884d976d74484 not found: ID does not exist"
Jul 30 20:20:59.584335 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.584314    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"]
Jul 30 20:20:59.587682 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:20:59.587661    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/sub-test-model-ecda00-kserve-747867bfb9-79j5m"]
Jul 30 20:21:00.785843 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:00.785810    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48fb8f43-4586-4552-9d06-cbc11852d887" path="/var/lib/kubelet/pods/48fb8f43-4586-4552-9d06-cbc11852d887/volumes"
Jul 30 20:21:04.568376 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:04.568340    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:21:06.201739 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:06.201703    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:21:06.202236 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:06.201796    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:21:06.214033 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:06.214013    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:21:06.603370 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:06.603346    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:21:23.866157 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.866125    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"]
Jul 30 20:21:23.866670 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.866395    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48fb8f43-4586-4552-9d06-cbc11852d887" containerName="main"
Jul 30 20:21:23.866670 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.866406    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="48fb8f43-4586-4552-9d06-cbc11852d887" containerName="main"
Jul 30 20:21:23.866670 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.866456    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="48fb8f43-4586-4552-9d06-cbc11852d887" containerName="main"
Jul 30 20:21:23.869228 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.869211    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:23.872636 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.872611    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"kube-root-ca.crt\""
Jul 30 20:21:23.872778 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.872688    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"rate-test-model-a769e5-kserve-self-signed-certs\""
Jul 30 20:21:23.872778 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.872611    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"openshift-service-ca.crt\""
Jul 30 20:21:23.872855 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.872611    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"default-dockercfg-gprzj\""
Jul 30 20:21:23.880551 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.880528    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"]
Jul 30 20:21:23.969877 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.969854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-dshm\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:23.969992 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.969885    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tmp-dir\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:23.969992 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.969906    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq9fb\" (UniqueName: \"kubernetes.io/projected/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-kube-api-access-sq9fb\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:23.970149 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.969993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-home\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:23.970149 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.970098    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-model-cache\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:23.970149 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:23.970135    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tls-certs\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.070865 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.070838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-dshm\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.070965 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.070869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tmp-dir\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.070965 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.070888    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq9fb\" (UniqueName: \"kubernetes.io/projected/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-kube-api-access-sq9fb\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.070965 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.070913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-home\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.071140 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.070971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-model-cache\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.071140 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.071003    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tls-certs\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.071393 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.071364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tmp-dir\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.071507 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.071403    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-home\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.071507 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.071440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-model-cache\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.073145 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.073126    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-dshm\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.073543 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.073524    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tls-certs\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.079172 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.079150    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq9fb\" (UniqueName: \"kubernetes.io/projected/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-kube-api-access-sq9fb\") pod \"rate-test-model-a769e5-kserve-6456d75475-kt4qt\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") " pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.178321 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.178243    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:24.302910 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.302885    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"]
Jul 30 20:21:24.305291 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:21:24.305262    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78d91e00_66b7_43ec_a3c7_f588f22f2fc9.slice/crio-5abc399dbf94b681d795ccef223984e6830cedf2188acc6ba3fdcceaa310e7a2 WatchSource:0}: Error finding container 5abc399dbf94b681d795ccef223984e6830cedf2188acc6ba3fdcceaa310e7a2: Status 404 returned error can't find the container with id 5abc399dbf94b681d795ccef223984e6830cedf2188acc6ba3fdcceaa310e7a2
Jul 30 20:21:24.669179 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.669145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt" event={"ID":"78d91e00-66b7-43ec-a3c7-f588f22f2fc9","Type":"ContainerStarted","Data":"900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd"}
Jul 30 20:21:24.669350 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.669189    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt" event={"ID":"78d91e00-66b7-43ec-a3c7-f588f22f2fc9","Type":"ContainerStarted","Data":"5abc399dbf94b681d795ccef223984e6830cedf2188acc6ba3fdcceaa310e7a2"}
Jul 30 20:21:24.686388 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:24.686346    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt" podStartSLOduration=1.686332086 podStartE2EDuration="1.686332086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:21:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:21:24.685386519 +0000 UTC m=+3404.446141776" watchObservedRunningTime="2026-07-30 20:21:24.686332086 +0000 UTC m=+3404.447087343"
Jul 30 20:21:27.282588 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:27.282561    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:21:34.178727 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:34.178639    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:34.179132 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:34.178919    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:34.190978 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:34.190951    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:21:34.720043 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:21:34.720019    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:22:14.965849 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:22:14.965819    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:22:28.970089 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:22:28.970044    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:22:57.788025 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:22:57.787991    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:23:04.402879 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:04.402799    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"]
Jul 30 20:23:05.282555 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:05.282519    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"]
Jul 30 20:23:12.370180 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:12.370143    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:23:19.421336 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.421293    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h" podUID="6bc56200-a70c-4d2f-8321-06aa160876f6" containerName="main" containerID="cri-o://90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf" gracePeriod=45
Jul 30 20:23:19.660979 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.660956    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:23:19.781065 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.781008    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bc56200-a70c-4d2f-8321-06aa160876f6-tls-certs\") pod \"6bc56200-a70c-4d2f-8321-06aa160876f6\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") "
Jul 30 20:23:19.781065 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.781043    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-dshm\") pod \"6bc56200-a70c-4d2f-8321-06aa160876f6\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") "
Jul 30 20:23:19.781065 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.781062    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-home\") pod \"6bc56200-a70c-4d2f-8321-06aa160876f6\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") "
Jul 30 20:23:19.781255 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.781111    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-model-cache\") pod \"6bc56200-a70c-4d2f-8321-06aa160876f6\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") "
Jul 30 20:23:19.781255 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.781175    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-tmp-dir\") pod \"6bc56200-a70c-4d2f-8321-06aa160876f6\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") "
Jul 30 20:23:19.781255 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.781199    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28jqt\" (UniqueName: \"kubernetes.io/projected/6bc56200-a70c-4d2f-8321-06aa160876f6-kube-api-access-28jqt\") pod \"6bc56200-a70c-4d2f-8321-06aa160876f6\" (UID: \"6bc56200-a70c-4d2f-8321-06aa160876f6\") "
Jul 30 20:23:19.781404 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.781378    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "6bc56200-a70c-4d2f-8321-06aa160876f6" (UID: "6bc56200-a70c-4d2f-8321-06aa160876f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:23:19.781466 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.781393    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-home" (OuterVolumeSpecName: "home") pod "6bc56200-a70c-4d2f-8321-06aa160876f6" (UID: "6bc56200-a70c-4d2f-8321-06aa160876f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:23:19.781524 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.781473    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6bc56200-a70c-4d2f-8321-06aa160876f6" (UID: "6bc56200-a70c-4d2f-8321-06aa160876f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:23:19.783125 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.783091    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bc56200-a70c-4d2f-8321-06aa160876f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6bc56200-a70c-4d2f-8321-06aa160876f6" (UID: "6bc56200-a70c-4d2f-8321-06aa160876f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:23:19.783261 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.783126    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-dshm" (OuterVolumeSpecName: "dshm") pod "6bc56200-a70c-4d2f-8321-06aa160876f6" (UID: "6bc56200-a70c-4d2f-8321-06aa160876f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:23:19.783326 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.783273    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc56200-a70c-4d2f-8321-06aa160876f6-kube-api-access-28jqt" (OuterVolumeSpecName: "kube-api-access-28jqt") pod "6bc56200-a70c-4d2f-8321-06aa160876f6" (UID: "6bc56200-a70c-4d2f-8321-06aa160876f6"). InnerVolumeSpecName "kube-api-access-28jqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:23:19.881771 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.881738    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bc56200-a70c-4d2f-8321-06aa160876f6-tls-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:19.881771 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.881768    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-dshm\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:19.881771 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.881778    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-home\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:19.881942 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.881786    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-model-cache\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:19.881942 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.881794    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bc56200-a70c-4d2f-8321-06aa160876f6-tmp-dir\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:19.881942 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:19.881801    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28jqt\" (UniqueName: \"kubernetes.io/projected/6bc56200-a70c-4d2f-8321-06aa160876f6-kube-api-access-28jqt\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:20.120346 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.120316    2577 generic.go:356] "Generic (PLEG): container finished" podID="6bc56200-a70c-4d2f-8321-06aa160876f6" containerID="90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf" exitCode=0
Jul 30 20:23:20.120451 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.120361    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"
Jul 30 20:23:20.120498 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.120358    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h" event={"ID":"6bc56200-a70c-4d2f-8321-06aa160876f6","Type":"ContainerDied","Data":"90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf"}
Jul 30 20:23:20.120531 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.120491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h" event={"ID":"6bc56200-a70c-4d2f-8321-06aa160876f6","Type":"ContainerDied","Data":"ab9424c110ca635835c6eee879f89fbfc7e7ecf9fb9a70da1b32463a1271ecc7"}
Jul 30 20:23:20.120531 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.120520    2577 scope.go:117] "RemoveContainer" containerID="90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf"
Jul 30 20:23:20.135646 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.135624    2577 scope.go:117] "RemoveContainer" containerID="90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf"
Jul 30 20:23:20.135927 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:23:20.135908    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf\": container with ID starting with 90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf not found: ID does not exist" containerID="90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf"
Jul 30 20:23:20.136002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.135939    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf"} err="failed to get container status \"90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf\": rpc error: code = NotFound desc = could not find container \"90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf\": container with ID starting with 90d6621c95b9afa0140ce2119317be78a4eb0767ec8dca000050cc5dad5fc0cf not found: ID does not exist"
Jul 30 20:23:20.143317 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.143295    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"]
Jul 30 20:23:20.145533 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.145512    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ecda00/rate-test-model-ecda00-kserve-75df8b4c67-vh28h"]
Jul 30 20:23:20.301420 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.301368    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt" podUID="78d91e00-66b7-43ec-a3c7-f588f22f2fc9" containerName="main" containerID="cri-o://900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd" gracePeriod=45
Jul 30 20:23:20.545361 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.545338    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:23:20.685526 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.685458    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq9fb\" (UniqueName: \"kubernetes.io/projected/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-kube-api-access-sq9fb\") pod \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") "
Jul 30 20:23:20.685526 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.685511    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-home\") pod \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") "
Jul 30 20:23:20.685687 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.685533    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-model-cache\") pod \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") "
Jul 30 20:23:20.685687 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.685561    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tls-certs\") pod \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") "
Jul 30 20:23:20.685687 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.685607    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tmp-dir\") pod \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") "
Jul 30 20:23:20.685687 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.685670    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-dshm\") pod \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\" (UID: \"78d91e00-66b7-43ec-a3c7-f588f22f2fc9\") "
Jul 30 20:23:20.685889 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.685761    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-home" (OuterVolumeSpecName: "home") pod "78d91e00-66b7-43ec-a3c7-f588f22f2fc9" (UID: "78d91e00-66b7-43ec-a3c7-f588f22f2fc9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:23:20.685889 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.685811    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-model-cache" (OuterVolumeSpecName: "model-cache") pod "78d91e00-66b7-43ec-a3c7-f588f22f2fc9" (UID: "78d91e00-66b7-43ec-a3c7-f588f22f2fc9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:23:20.685995 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.685923    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-home\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:20.685995 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.685948    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "78d91e00-66b7-43ec-a3c7-f588f22f2fc9" (UID: "78d91e00-66b7-43ec-a3c7-f588f22f2fc9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:23:20.687760 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.687740    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-kube-api-access-sq9fb" (OuterVolumeSpecName: "kube-api-access-sq9fb") pod "78d91e00-66b7-43ec-a3c7-f588f22f2fc9" (UID: "78d91e00-66b7-43ec-a3c7-f588f22f2fc9"). InnerVolumeSpecName "kube-api-access-sq9fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:23:20.687839 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.687760    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-dshm" (OuterVolumeSpecName: "dshm") pod "78d91e00-66b7-43ec-a3c7-f588f22f2fc9" (UID: "78d91e00-66b7-43ec-a3c7-f588f22f2fc9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:23:20.687839 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.687812    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "78d91e00-66b7-43ec-a3c7-f588f22f2fc9" (UID: "78d91e00-66b7-43ec-a3c7-f588f22f2fc9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:23:20.783851 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.783825    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bc56200-a70c-4d2f-8321-06aa160876f6" path="/var/lib/kubelet/pods/6bc56200-a70c-4d2f-8321-06aa160876f6/volumes"
Jul 30 20:23:20.786216 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.786200    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-dshm\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:20.786270 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.786219    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq9fb\" (UniqueName: \"kubernetes.io/projected/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-kube-api-access-sq9fb\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:20.786270 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.786230    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-model-cache\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:20.786270 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.786239    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tls-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:20.786270 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:20.786262    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78d91e00-66b7-43ec-a3c7-f588f22f2fc9-tmp-dir\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:21.126243 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.126219    2577 generic.go:356] "Generic (PLEG): container finished" podID="78d91e00-66b7-43ec-a3c7-f588f22f2fc9" containerID="900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd" exitCode=0
Jul 30 20:23:21.126340 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.126248    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt" event={"ID":"78d91e00-66b7-43ec-a3c7-f588f22f2fc9","Type":"ContainerDied","Data":"900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd"}
Jul 30 20:23:21.126340 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.126259    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"
Jul 30 20:23:21.126340 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.126272    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt" event={"ID":"78d91e00-66b7-43ec-a3c7-f588f22f2fc9","Type":"ContainerDied","Data":"5abc399dbf94b681d795ccef223984e6830cedf2188acc6ba3fdcceaa310e7a2"}
Jul 30 20:23:21.126340 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.126287    2577 scope.go:117] "RemoveContainer" containerID="900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd"
Jul 30 20:23:21.134065 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.134046    2577 scope.go:117] "RemoveContainer" containerID="900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd"
Jul 30 20:23:21.134333 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:23:21.134300    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd\": container with ID starting with 900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd not found: ID does not exist" containerID="900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd"
Jul 30 20:23:21.134402 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.134337    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd"} err="failed to get container status \"900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd\": rpc error: code = NotFound desc = could not find container \"900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd\": container with ID starting with 900ca1f60c2b30142fd2fdde9cc543dd3e6d5f17cf3721d87afc5e298ede91dd not found: ID does not exist"
Jul 30 20:23:21.141933 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.141912    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"]
Jul 30 20:23:21.149563 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.149543    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/rate-test-model-a769e5-kserve-6456d75475-kt4qt"]
Jul 30 20:23:21.356358 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.356336    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7730-140683917180976"]
Jul 30 20:23:21.356586 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.356574    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc56200-a70c-4d2f-8321-06aa160876f6" containerName="main"
Jul 30 20:23:21.356626 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.356588    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc56200-a70c-4d2f-8321-06aa160876f6" containerName="main"
Jul 30 20:23:21.356626 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.356610    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78d91e00-66b7-43ec-a3c7-f588f22f2fc9" containerName="main"
Jul 30 20:23:21.356626 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.356615    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="78d91e00-66b7-43ec-a3c7-f588f22f2fc9" containerName="main"
Jul 30 20:23:21.356725 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.356660    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bc56200-a70c-4d2f-8321-06aa160876f6" containerName="main"
Jul 30 20:23:21.356725 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.356668    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="78d91e00-66b7-43ec-a3c7-f588f22f2fc9" containerName="main"
Jul 30 20:23:21.360540 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.360527    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:21.362957 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.362795    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4wwrf\""
Jul 30 20:23:21.364123 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.364038    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917180976"]
Jul 30 20:23:21.491519 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.491449    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8kjq\" (UniqueName: \"kubernetes.io/projected/02b92013-ceb2-4aa3-a533-daf310c02c73-kube-api-access-t8kjq\") pod \"test-curl-7730-140683917180976\" (UID: \"02b92013-ceb2-4aa3-a533-daf310c02c73\") " pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:21.592660 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.592637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8kjq\" (UniqueName: \"kubernetes.io/projected/02b92013-ceb2-4aa3-a533-daf310c02c73-kube-api-access-t8kjq\") pod \"test-curl-7730-140683917180976\" (UID: \"02b92013-ceb2-4aa3-a533-daf310c02c73\") " pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:21.601484 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.601466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8kjq\" (UniqueName: \"kubernetes.io/projected/02b92013-ceb2-4aa3-a533-daf310c02c73-kube-api-access-t8kjq\") pod \"test-curl-7730-140683917180976\" (UID: \"02b92013-ceb2-4aa3-a533-daf310c02c73\") " pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:21.670181 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.670159    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:21.791877 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.791843    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917180976"]
Jul 30 20:23:21.795018 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:23:21.794992    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02b92013_ceb2_4aa3_a533_daf310c02c73.slice/crio-a9392b812ec62b86b10f6d5c51793c8652644fa53c3122e5607b52fc8c30a581 WatchSource:0}: Error finding container a9392b812ec62b86b10f6d5c51793c8652644fa53c3122e5607b52fc8c30a581: Status 404 returned error can't find the container with id a9392b812ec62b86b10f6d5c51793c8652644fa53c3122e5607b52fc8c30a581
Jul 30 20:23:21.796665 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:21.796648    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:23:22.133385 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:22.133349    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917180976" event={"ID":"02b92013-ceb2-4aa3-a533-daf310c02c73","Type":"ContainerStarted","Data":"a9392b812ec62b86b10f6d5c51793c8652644fa53c3122e5607b52fc8c30a581"}
Jul 30 20:23:22.783970 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:22.783913    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78d91e00-66b7-43ec-a3c7-f588f22f2fc9" path="/var/lib/kubelet/pods/78d91e00-66b7-43ec-a3c7-f588f22f2fc9/volumes"
Jul 30 20:23:23.138590 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:23.138562    2577 generic.go:356] "Generic (PLEG): container finished" podID="02b92013-ceb2-4aa3-a533-daf310c02c73" containerID="1cf4a05277023b95c54ef3fb8f4e5b881ee5a073a087b4592b75f3bc79e34fcd" exitCode=0
Jul 30 20:23:23.138704 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:23.138644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917180976" event={"ID":"02b92013-ceb2-4aa3-a533-daf310c02c73","Type":"ContainerDied","Data":"1cf4a05277023b95c54ef3fb8f4e5b881ee5a073a087b4592b75f3bc79e34fcd"}
Jul 30 20:23:24.270824 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:24.270798    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:24.304061 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:24.304026    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7730-140683917180976_02b92013-ceb2-4aa3-a533-daf310c02c73/test-curl-7730-140683917180976/0.log"
Jul 30 20:23:24.326982 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:24.326958    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7730-140683917180976"]
Jul 30 20:23:24.328427 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:24.328406    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7730-140683917180976"]
Jul 30 20:23:24.411417 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:24.411352    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8kjq\" (UniqueName: \"kubernetes.io/projected/02b92013-ceb2-4aa3-a533-daf310c02c73-kube-api-access-t8kjq\") pod \"02b92013-ceb2-4aa3-a533-daf310c02c73\" (UID: \"02b92013-ceb2-4aa3-a533-daf310c02c73\") "
Jul 30 20:23:24.413574 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:24.413550    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02b92013-ceb2-4aa3-a533-daf310c02c73-kube-api-access-t8kjq" (OuterVolumeSpecName: "kube-api-access-t8kjq") pod "02b92013-ceb2-4aa3-a533-daf310c02c73" (UID: "02b92013-ceb2-4aa3-a533-daf310c02c73"). InnerVolumeSpecName "kube-api-access-t8kjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:23:24.512287 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:24.512258    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8kjq\" (UniqueName: \"kubernetes.io/projected/02b92013-ceb2-4aa3-a533-daf310c02c73-kube-api-access-t8kjq\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:23:24.783843 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:24.783776    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02b92013-ceb2-4aa3-a533-daf310c02c73" path="/var/lib/kubelet/pods/02b92013-ceb2-4aa3-a533-daf310c02c73/volumes"
Jul 30 20:23:25.155617 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.155592    2577 scope.go:117] "RemoveContainer" containerID="1cf4a05277023b95c54ef3fb8f4e5b881ee5a073a087b4592b75f3bc79e34fcd"
Jul 30 20:23:25.155782 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.155599    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:25.186632 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.186608    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"]
Jul 30 20:23:25.186890 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.186878    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02b92013-ceb2-4aa3-a533-daf310c02c73" containerName="test-curl-7730-140683917180976"
Jul 30 20:23:25.186932 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.186892    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="02b92013-ceb2-4aa3-a533-daf310c02c73" containerName="test-curl-7730-140683917180976"
Jul 30 20:23:25.186964 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.186947    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="02b92013-ceb2-4aa3-a533-daf310c02c73" containerName="test-curl-7730-140683917180976"
Jul 30 20:23:25.190748 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.190731    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.195180 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.195157    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-0c599c-openshift-default-dockercfg-9r68z\""
Jul 30 20:23:25.209895 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.209873    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"]
Jul 30 20:23:25.318710 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.318689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istiod-ca-cert\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.319042 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.318725    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-credential-socket\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.319042 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.318744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-socket\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.319042 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.318792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-data\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.319042 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.318833    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-envoy\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.319042 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.318859    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-token\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.319042 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.318886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-podinfo\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.319042 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.318907    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kftvd\" (UniqueName: \"kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-kube-api-access-kftvd\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.319042 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.318932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-certs\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420040 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.419966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-envoy\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420040 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-token\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420236 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-podinfo\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420236 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kftvd\" (UniqueName: \"kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-kube-api-access-kftvd\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420236 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-certs\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420236 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420168    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istiod-ca-cert\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420236 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-credential-socket\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420502 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-socket\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420502 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-data\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420607 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420535    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-certs\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420607 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-credential-socket\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420724 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420707    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-socket\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420809 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420787    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-data\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.420915 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.420897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istiod-ca-cert\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.422716 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.422692    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-envoy\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.422857 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.422841    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-podinfo\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.427853 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.427828    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kftvd\" (UniqueName: \"kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-kube-api-access-kftvd\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.427853 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.427848    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-token\") pod \"e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") " pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.500404 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.500381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:25.628439 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.628414    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"]
Jul 30 20:23:25.630550 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:23:25.630521    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d116a9_f27c_4e2d_b1aa_ed0453e30551.slice/crio-764b3873ceda0b89fba9bcc786b1efdebe2aeafaeaab075b18717d99e8a29a4f WatchSource:0}: Error finding container 764b3873ceda0b89fba9bcc786b1efdebe2aeafaeaab075b18717d99e8a29a4f: Status 404 returned error can't find the container with id 764b3873ceda0b89fba9bcc786b1efdebe2aeafaeaab075b18717d99e8a29a4f
Jul 30 20:23:25.632791 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.632759    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:23:25.632875 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.632824    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:23:25.632875 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:25.632855    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:23:26.161828 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:26.161793    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" event={"ID":"c3d116a9-f27c-4e2d-b1aa-ed0453e30551","Type":"ContainerStarted","Data":"80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022"}
Jul 30 20:23:26.161987 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:26.161842    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" event={"ID":"c3d116a9-f27c-4e2d-b1aa-ed0453e30551","Type":"ContainerStarted","Data":"764b3873ceda0b89fba9bcc786b1efdebe2aeafaeaab075b18717d99e8a29a4f"}
Jul 30 20:23:26.180861 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:26.180814    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" podStartSLOduration=1.180799591 podStartE2EDuration="1.180799591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:23:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:23:26.179409017 +0000 UTC m=+3525.940164276" watchObservedRunningTime="2026-07-30 20:23:26.180799591 +0000 UTC m=+3525.941555054"
Jul 30 20:23:26.500614 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:26.500546    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:26.505056 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:26.505032    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:27.165307 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:27.165278    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:27.166262 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:27.166242    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:29.180056 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.180026    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"]
Jul 30 20:23:29.182815 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.182798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:29.185431 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.185410    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-0c599c-serving-cert\""
Jul 30 20:23:29.194536 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.194512    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"]
Jul 30 20:23:29.250933 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.250908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663d5460-ac42-4131-b9d4-c43c80bdf6de-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:29.251105 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.250967    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/663d5460-ac42-4131-b9d4-c43c80bdf6de-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:29.251105 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.251033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmtdc\" (UniqueName: \"kubernetes.io/projected/663d5460-ac42-4131-b9d4-c43c80bdf6de-kube-api-access-lmtdc\") pod \"maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:29.352052 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.352025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/663d5460-ac42-4131-b9d4-c43c80bdf6de-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:29.352204 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.352094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmtdc\" (UniqueName: \"kubernetes.io/projected/663d5460-ac42-4131-b9d4-c43c80bdf6de-kube-api-access-lmtdc\") pod \"maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:29.352204 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.352166    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663d5460-ac42-4131-b9d4-c43c80bdf6de-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:29.352327 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:23:29.352288    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-serving-cert: secret "maas-api-e2e-ipp-a-0c599c-serving-cert" not found
Jul 30 20:23:29.352393 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:23:29.352361    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/663d5460-ac42-4131-b9d4-c43c80bdf6de-maas-api-tls podName:663d5460-ac42-4131-b9d4-c43c80bdf6de nodeName:}" failed. No retries permitted until 2026-07-30 20:23:29.852337632 +0000 UTC m=+3529.613092870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/663d5460-ac42-4131-b9d4-c43c80bdf6de-maas-api-tls") pod "maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k" (UID: "663d5460-ac42-4131-b9d4-c43c80bdf6de") : secret "maas-api-e2e-ipp-a-0c599c-serving-cert" not found
Jul 30 20:23:29.352640 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.352619    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/663d5460-ac42-4131-b9d4-c43c80bdf6de-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:29.362882 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.362860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmtdc\" (UniqueName: \"kubernetes.io/projected/663d5460-ac42-4131-b9d4-c43c80bdf6de-kube-api-access-lmtdc\") pod \"maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:29.854669 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.854624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663d5460-ac42-4131-b9d4-c43c80bdf6de-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:29.857107 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:29.857064    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663d5460-ac42-4131-b9d4-c43c80bdf6de-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:30.095642 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:30.095613    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:30.226191 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:30.226167    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"]
Jul 30 20:23:30.228373 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:23:30.228347    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663d5460_ac42_4131_b9d4_c43c80bdf6de.slice/crio-a331b0482e690bd04f6058efcaaf1e871dc1554091fe3b8562334cf5fe35ea6a WatchSource:0}: Error finding container a331b0482e690bd04f6058efcaaf1e871dc1554091fe3b8562334cf5fe35ea6a: Status 404 returned error can't find the container with id a331b0482e690bd04f6058efcaaf1e871dc1554091fe3b8562334cf5fe35ea6a
Jul 30 20:23:31.183255 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:31.183222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k" event={"ID":"663d5460-ac42-4131-b9d4-c43c80bdf6de","Type":"ContainerStarted","Data":"f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc"}
Jul 30 20:23:31.183441 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:31.183280    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k" event={"ID":"663d5460-ac42-4131-b9d4-c43c80bdf6de","Type":"ContainerStarted","Data":"a331b0482e690bd04f6058efcaaf1e871dc1554091fe3b8562334cf5fe35ea6a"}
Jul 30 20:23:31.183441 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:31.183307    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:31.200941 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:31.200893    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k" podStartSLOduration=1.6383071550000001 podStartE2EDuration="2.200878927s" totalImagesPullingTime="562.571772ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:23:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:23:31.199721355 +0000 UTC m=+3530.960476614" watchObservedRunningTime="2026-07-30 20:23:31.200878927 +0000 UTC m=+3530.961634184"
Jul 30 20:23:36.676782 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:36.676749    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:23:37.193317 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:37.193286    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:23:43.207236 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:43.207189    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 20:23:43.207643 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:43.207255    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 20:23:45.250366 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:45.250337    2577 generic.go:356] "Generic (PLEG): container finished" podID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerID="80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022" exitCode=0
Jul 30 20:23:45.250731 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:45.250381    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" event={"ID":"c3d116a9-f27c-4e2d-b1aa-ed0453e30551","Type":"ContainerDied","Data":"80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022"}
Jul 30 20:23:45.250731 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:45.250722    2577 scope.go:117] "RemoveContainer" containerID="80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022"
Jul 30 20:23:45.251720 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:45.251693    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:23:45.251773 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:45.251758    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:23:45.251816 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:45.251788    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:23:46.257870 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:46.257832    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" event={"ID":"c3d116a9-f27c-4e2d-b1aa-ed0453e30551","Type":"ContainerStarted","Data":"1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0"}
Jul 30 20:23:46.501028 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:46.500986    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:46.501248 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:46.501229    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 30 20:23:46.501329 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:46.501284    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 30 20:23:47.501199 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:47.501164    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 30 20:23:47.501604 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:47.501219    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 30 20:23:48.501083 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:48.501031    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 30 20:23:48.501253 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:48.501110    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 30 20:23:49.501417 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:49.501376    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 30 20:23:49.501820 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:49.501441    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 30 20:23:50.504470 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:50.504433    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:50.504957 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:50.504810    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:50.505374 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:50.505353    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:23:54.052019 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.051983    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"]
Jul 30 20:23:54.056937 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.056918    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.059432 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.059410    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-d67d52-openshift-default-dockercfg-jhwkq\""
Jul 30 20:23:54.068580 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.068557    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"]
Jul 30 20:23:54.144820 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.144799    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k8fr\" (UniqueName: \"kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-kube-api-access-7k8fr\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.144928 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.144830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-socket\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.144928 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.144854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-envoy\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.144928 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.144895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-data\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.145040 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.144932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-credential-socket\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.145040 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.144954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9abe6e50-80e8-4321-9093-ffee778c4cdf-istiod-ca-cert\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.145040 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.144986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-token\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.145154 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.145054    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-podinfo\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.145154 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.145106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-certs\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.245840 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.245813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k8fr\" (UniqueName: \"kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-kube-api-access-7k8fr\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.245968 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.245853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-socket\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.245968 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.245888    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-envoy\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.245968 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.245916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-data\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.245968 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.245948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-credential-socket\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.246202 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.245974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9abe6e50-80e8-4321-9093-ffee778c4cdf-istiod-ca-cert\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.246202 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.246017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-token\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.246202 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.246089    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-podinfo\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.246202 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.246121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-certs\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.246465 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.246433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-credential-socket\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.246580 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.246486    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-data\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.246580 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.246529    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-certs\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.246580 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.246548    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-socket\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.246816 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.246794    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9abe6e50-80e8-4321-9093-ffee778c4cdf-istiod-ca-cert\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.248213 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.248196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-envoy\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.248553 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.248533    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-podinfo\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.253575 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.253552    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-token\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.253746 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.253725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k8fr\" (UniqueName: \"kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-kube-api-access-7k8fr\") pod \"e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") " pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.366241 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.366190    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:54.514285 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.514258    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"]
Jul 30 20:23:54.516046 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:23:54.516013    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9abe6e50_80e8_4321_9093_ffee778c4cdf.slice/crio-39064a0dc0e7ccf53e697cffda76f42015b6a0c4d7d3a7e54e10a52d61138a5f WatchSource:0}: Error finding container 39064a0dc0e7ccf53e697cffda76f42015b6a0c4d7d3a7e54e10a52d61138a5f: Status 404 returned error can't find the container with id 39064a0dc0e7ccf53e697cffda76f42015b6a0c4d7d3a7e54e10a52d61138a5f
Jul 30 20:23:54.518349 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.518317    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:23:54.518466 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.518403    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:23:54.518466 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:54.518447    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:23:55.293397 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:55.293356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" event={"ID":"9abe6e50-80e8-4321-9093-ffee778c4cdf","Type":"ContainerStarted","Data":"5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee"}
Jul 30 20:23:55.293757 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:55.293419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" event={"ID":"9abe6e50-80e8-4321-9093-ffee778c4cdf","Type":"ContainerStarted","Data":"39064a0dc0e7ccf53e697cffda76f42015b6a0c4d7d3a7e54e10a52d61138a5f"}
Jul 30 20:23:55.315762 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:55.315716    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" podStartSLOduration=1.315701808 podStartE2EDuration="1.315701808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:23:55.31365642 +0000 UTC m=+3555.074411689" watchObservedRunningTime="2026-07-30 20:23:55.315701808 +0000 UTC m=+3555.076457139"
Jul 30 20:23:55.366361 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:55.366334    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:55.370775 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:55.370755    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:56.298173 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:56.298138    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:23:56.299050 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:23:56.299029    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:24:06.966747 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:06.966713    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:24:15.382890 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:15.382861    2577 generic.go:356] "Generic (PLEG): container finished" podID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerID="5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee" exitCode=0
Jul 30 20:24:15.383263 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:15.382927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" event={"ID":"9abe6e50-80e8-4321-9093-ffee778c4cdf","Type":"ContainerDied","Data":"5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee"}
Jul 30 20:24:15.383307 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:15.383261    2577 scope.go:117] "RemoveContainer" containerID="5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee"
Jul 30 20:24:15.384327 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:15.384298    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:24:15.384378 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:15.384367    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:24:15.384415 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:15.384405    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 20:24:16.389169 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:16.389133    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" event={"ID":"9abe6e50-80e8-4321-9093-ffee778c4cdf","Type":"ContainerStarted","Data":"90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379"}
Jul 30 20:24:17.366839 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:17.366805    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:24:17.367051 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:17.367026    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 30 20:24:17.367127 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:17.367096    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 30 20:24:18.367401 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:18.367361    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 30 20:24:18.367813 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:18.367421    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 30 20:24:19.367466 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:19.367430    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 30 20:24:19.367945 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:19.367486    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 30 20:24:20.370201 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:20.370171    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:24:20.370645 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:20.370412    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:24:20.371175 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:20.371158    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:24:43.468596 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.468554    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"]
Jul 30 20:24:43.470988 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.470973    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.474234 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.474208    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0c599c\"/\"ipp-route-0c599c-kserve-self-signed-certs\""
Jul 30 20:24:43.474373 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.474305    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0c599c\"/\"openshift-service-ca.crt\""
Jul 30 20:24:43.474373 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.474310    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0c599c\"/\"kube-root-ca.crt\""
Jul 30 20:24:43.474481 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.474322    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0c599c\"/\"default-dockercfg-hpthv\""
Jul 30 20:24:43.480206 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.480182    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"]
Jul 30 20:24:43.616195 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.616170    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-model-cache\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.616328 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.616211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-dshm\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.616328 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.616232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-home\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.616328 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.616272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69mpd\" (UniqueName: \"kubernetes.io/projected/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-kube-api-access-69mpd\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.616328 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.616294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tls-certs\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.616471 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.616378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tmp-dir\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.716765 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.716729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69mpd\" (UniqueName: \"kubernetes.io/projected/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-kube-api-access-69mpd\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.716958 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.716771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tls-certs\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.716958 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.716821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tmp-dir\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.716958 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.716845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-model-cache\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.716958 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.716889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-dshm\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.716958 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.716914    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-home\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.717361 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.717337    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tmp-dir\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.717554 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.717533    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-model-cache\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.717624 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.717592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-home\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.719438 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.719373    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-dshm\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.719631 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.719613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tls-certs\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.724582 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.724564    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69mpd\" (UniqueName: \"kubernetes.io/projected/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-kube-api-access-69mpd\") pod \"ipp-route-0c599c-kserve-659d76f875-5qt6v\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") " pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.780806 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.780771    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:43.907883 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:43.907849    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"]
Jul 30 20:24:43.911164 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:24:43.911130    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf49f8aa1_cdbb_403a_b578_20ad89bd3ed4.slice/crio-6d6bca439f9d7c680688383a37c4bac17ea64b33528cd8df60232d6af6e2caf4 WatchSource:0}: Error finding container 6d6bca439f9d7c680688383a37c4bac17ea64b33528cd8df60232d6af6e2caf4: Status 404 returned error can't find the container with id 6d6bca439f9d7c680688383a37c4bac17ea64b33528cd8df60232d6af6e2caf4
Jul 30 20:24:44.498722 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:44.498686    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v" event={"ID":"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4","Type":"ContainerStarted","Data":"0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c"}
Jul 30 20:24:44.499119 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:44.498737    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v" event={"ID":"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4","Type":"ContainerStarted","Data":"6d6bca439f9d7c680688383a37c4bac17ea64b33528cd8df60232d6af6e2caf4"}
Jul 30 20:24:44.517172 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:44.517130    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v" podStartSLOduration=1.5171176929999999 podStartE2EDuration="1.517117693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:24:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:24:44.51453799 +0000 UTC m=+3604.275293248" watchObservedRunningTime="2026-07-30 20:24:44.517117693 +0000 UTC m=+3604.277872950"
Jul 30 20:24:46.767309 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:46.767277    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:24:53.780904 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:53.780871    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:53.781343 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:53.780924    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:53.793167 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:53.793139    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:24:54.549606 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:24:54.549578    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:25:14.967783 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:25:14.967752    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:25:40.465116 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:25:40.465063    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:26:05.757982 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:05.757898    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:26:25.870551 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:25.870515    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:26:27.000754 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.000724    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"]
Jul 30 20:26:27.001251 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.000975    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k" podUID="663d5460-ac42-4131-b9d4-c43c80bdf6de" containerName="maas-api" containerID="cri-o://f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc" gracePeriod=30
Jul 30 20:26:27.233928 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.233901    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:26:27.300852 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.300825    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663d5460-ac42-4131-b9d4-c43c80bdf6de-maas-api-tls\") pod \"663d5460-ac42-4131-b9d4-c43c80bdf6de\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") "
Jul 30 20:26:27.301019 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.300886    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/663d5460-ac42-4131-b9d4-c43c80bdf6de-openshift-service-ca\") pod \"663d5460-ac42-4131-b9d4-c43c80bdf6de\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") "
Jul 30 20:26:27.301019 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.300924    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmtdc\" (UniqueName: \"kubernetes.io/projected/663d5460-ac42-4131-b9d4-c43c80bdf6de-kube-api-access-lmtdc\") pod \"663d5460-ac42-4131-b9d4-c43c80bdf6de\" (UID: \"663d5460-ac42-4131-b9d4-c43c80bdf6de\") "
Jul 30 20:26:27.301322 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.301297    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/663d5460-ac42-4131-b9d4-c43c80bdf6de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "663d5460-ac42-4131-b9d4-c43c80bdf6de" (UID: "663d5460-ac42-4131-b9d4-c43c80bdf6de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:26:27.303200 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.303167    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/663d5460-ac42-4131-b9d4-c43c80bdf6de-kube-api-access-lmtdc" (OuterVolumeSpecName: "kube-api-access-lmtdc") pod "663d5460-ac42-4131-b9d4-c43c80bdf6de" (UID: "663d5460-ac42-4131-b9d4-c43c80bdf6de"). InnerVolumeSpecName "kube-api-access-lmtdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:26:27.303309 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.303231    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/663d5460-ac42-4131-b9d4-c43c80bdf6de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "663d5460-ac42-4131-b9d4-c43c80bdf6de" (UID: "663d5460-ac42-4131-b9d4-c43c80bdf6de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:26:27.402039 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.402007    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663d5460-ac42-4131-b9d4-c43c80bdf6de-maas-api-tls\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:26:27.402161 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.402043    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/663d5460-ac42-4131-b9d4-c43c80bdf6de-openshift-service-ca\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:26:27.402161 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.402059    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmtdc\" (UniqueName: \"kubernetes.io/projected/663d5460-ac42-4131-b9d4-c43c80bdf6de-kube-api-access-lmtdc\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:26:27.921817 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.921787    2577 generic.go:356] "Generic (PLEG): container finished" podID="663d5460-ac42-4131-b9d4-c43c80bdf6de" containerID="f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc" exitCode=0
Jul 30 20:26:27.921979 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.921820    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"
Jul 30 20:26:27.921979 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.921869    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k" event={"ID":"663d5460-ac42-4131-b9d4-c43c80bdf6de","Type":"ContainerDied","Data":"f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc"}
Jul 30 20:26:27.921979 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.921921    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k" event={"ID":"663d5460-ac42-4131-b9d4-c43c80bdf6de","Type":"ContainerDied","Data":"a331b0482e690bd04f6058efcaaf1e871dc1554091fe3b8562334cf5fe35ea6a"}
Jul 30 20:26:27.921979 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.921942    2577 scope.go:117] "RemoveContainer" containerID="f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc"
Jul 30 20:26:27.930537 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.930522    2577 scope.go:117] "RemoveContainer" containerID="f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc"
Jul 30 20:26:27.930765 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:26:27.930746    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc\": container with ID starting with f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc not found: ID does not exist" containerID="f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc"
Jul 30 20:26:27.930814 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.930774    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc"} err="failed to get container status \"f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc\": rpc error: code = NotFound desc = could not find container \"f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc\": container with ID starting with f7299a130e54ca3b6d226e5aa49a4cb1e32dcf23362c04ca67a7a726795619fc not found: ID does not exist"
Jul 30 20:26:27.941926 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.941904    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"]
Jul 30 20:26:27.945912 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:27.945893    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k"]
Jul 30 20:26:28.783616 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:28.783585    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="663d5460-ac42-4131-b9d4-c43c80bdf6de" path="/var/lib/kubelet/pods/663d5460-ac42-4131-b9d4-c43c80bdf6de/volumes"
Jul 30 20:26:30.189365 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:30.189327    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c599c-7757474969-v6d4k" podUID="663d5460-ac42-4131-b9d4-c43c80bdf6de" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.46:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 30 20:26:32.605031 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:32.605000    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"]
Jul 30 20:26:36.461673 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:36.461638    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:26:47.623854 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.623786    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v" podUID="f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" containerName="main" containerID="cri-o://0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c" gracePeriod=45
Jul 30 20:26:47.863599 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.863575    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:26:47.953215 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.953147    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tls-certs\") pod \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") "
Jul 30 20:26:47.953215 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.953202    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-home\") pod \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") "
Jul 30 20:26:47.953404 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.953231    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tmp-dir\") pod \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") "
Jul 30 20:26:47.953404 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.953289    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-model-cache\") pod \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") "
Jul 30 20:26:47.953404 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.953329    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69mpd\" (UniqueName: \"kubernetes.io/projected/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-kube-api-access-69mpd\") pod \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") "
Jul 30 20:26:47.953404 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.953354    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-dshm\") pod \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\" (UID: \"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4\") "
Jul 30 20:26:47.953623 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.953478    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-home" (OuterVolumeSpecName: "home") pod "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" (UID: "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:26:47.953623 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.953518    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" (UID: "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:26:47.953721 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.953613    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-model-cache" (OuterVolumeSpecName: "model-cache") pod "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" (UID: "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:26:47.953721 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.953640    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-home\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:26:47.953721 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.953659    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tmp-dir\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:26:47.955335 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.955303    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" (UID: "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:26:47.955335 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.955309    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-dshm" (OuterVolumeSpecName: "dshm") pod "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" (UID: "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:26:47.955472 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:47.955381    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-kube-api-access-69mpd" (OuterVolumeSpecName: "kube-api-access-69mpd") pod "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" (UID: "f49f8aa1-cdbb-403a-b578-20ad89bd3ed4"). InnerVolumeSpecName "kube-api-access-69mpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:26:48.005183 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.005157    2577 generic.go:356] "Generic (PLEG): container finished" podID="f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" containerID="0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c" exitCode=0
Jul 30 20:26:48.005288 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.005198    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"
Jul 30 20:26:48.005288 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.005223    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v" event={"ID":"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4","Type":"ContainerDied","Data":"0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c"}
Jul 30 20:26:48.005288 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.005253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v" event={"ID":"f49f8aa1-cdbb-403a-b578-20ad89bd3ed4","Type":"ContainerDied","Data":"6d6bca439f9d7c680688383a37c4bac17ea64b33528cd8df60232d6af6e2caf4"}
Jul 30 20:26:48.005288 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.005270    2577 scope.go:117] "RemoveContainer" containerID="0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c"
Jul 30 20:26:48.013096 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.013064    2577 scope.go:117] "RemoveContainer" containerID="0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c"
Jul 30 20:26:48.013353 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:26:48.013335    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c\": container with ID starting with 0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c not found: ID does not exist" containerID="0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c"
Jul 30 20:26:48.013408 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.013362    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c"} err="failed to get container status \"0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c\": rpc error: code = NotFound desc = could not find container \"0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c\": container with ID starting with 0b93e525efc19c52815640b188685def38443fd85a5d4709d68974ffba42b51c not found: ID does not exist"
Jul 30 20:26:48.024734 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.024713    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"]
Jul 30 20:26:48.028293 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.028272    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-0c599c/ipp-route-0c599c-kserve-659d76f875-5qt6v"]
Jul 30 20:26:48.054240 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.054222    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-tls-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:26:48.054316 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.054243    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-model-cache\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:26:48.054316 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.054252    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69mpd\" (UniqueName: \"kubernetes.io/projected/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-kube-api-access-69mpd\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:26:48.054316 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.054260    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4-dshm\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:26:48.783820 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:26:48.783781    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" path="/var/lib/kubelet/pods/f49f8aa1-cdbb-403a-b578-20ad89bd3ed4/volumes"
Jul 30 20:27:29.766331 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:29.766293    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"]
Jul 30 20:27:29.766770 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:29.766572    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy" containerID="cri-o://1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0" gracePeriod=30
Jul 30 20:27:35.004919 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.004892    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:27:35.087537 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.087516    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kftvd\" (UniqueName: \"kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-kube-api-access-kftvd\") pod \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") "
Jul 30 20:27:35.087663 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.087546    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-certs\") pod \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") "
Jul 30 20:27:35.087663 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.087579    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-token\") pod \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") "
Jul 30 20:27:35.087663 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.087623    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-envoy\") pod \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") "
Jul 30 20:27:35.087849 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.087693    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-podinfo\") pod \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") "
Jul 30 20:27:35.087849 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.087726    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-socket\") pod \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") "
Jul 30 20:27:35.087849 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.087756    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-credential-socket\") pod \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") "
Jul 30 20:27:35.087849 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.087783    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-data\") pod \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") "
Jul 30 20:27:35.087849 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.087815    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istiod-ca-cert\") pod \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\" (UID: \"c3d116a9-f27c-4e2d-b1aa-ed0453e30551\") "
Jul 30 20:27:35.087849 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.087804    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3d116a9-f27c-4e2d-b1aa-ed0453e30551" (UID: "c3d116a9-f27c-4e2d-b1aa-ed0453e30551"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:27:35.088195 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.088010    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3d116a9-f27c-4e2d-b1aa-ed0453e30551" (UID: "c3d116a9-f27c-4e2d-b1aa-ed0453e30551"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:27:35.088195 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.088051    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:35.088195 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.088062    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3d116a9-f27c-4e2d-b1aa-ed0453e30551" (UID: "c3d116a9-f27c-4e2d-b1aa-ed0453e30551"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:27:35.088361 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.088338    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3d116a9-f27c-4e2d-b1aa-ed0453e30551" (UID: "c3d116a9-f27c-4e2d-b1aa-ed0453e30551"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:27:35.088838 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.088805    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3d116a9-f27c-4e2d-b1aa-ed0453e30551" (UID: "c3d116a9-f27c-4e2d-b1aa-ed0453e30551"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:27:35.089907 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.089877    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3d116a9-f27c-4e2d-b1aa-ed0453e30551" (UID: "c3d116a9-f27c-4e2d-b1aa-ed0453e30551"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:27:35.090007 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.089942    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-kube-api-access-kftvd" (OuterVolumeSpecName: "kube-api-access-kftvd") pod "c3d116a9-f27c-4e2d-b1aa-ed0453e30551" (UID: "c3d116a9-f27c-4e2d-b1aa-ed0453e30551"). InnerVolumeSpecName "kube-api-access-kftvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:27:35.090007 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.089962    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3d116a9-f27c-4e2d-b1aa-ed0453e30551" (UID: "c3d116a9-f27c-4e2d-b1aa-ed0453e30551"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:27:35.090147 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.090021    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3d116a9-f27c-4e2d-b1aa-ed0453e30551" (UID: "c3d116a9-f27c-4e2d-b1aa-ed0453e30551"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 20:27:35.188338 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.188317    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-envoy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:35.188338 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.188338    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-podinfo\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:35.188464 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.188347    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-workload-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:35.188464 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.188355    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-credential-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:35.188464 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.188364    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-data\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:35.188464 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.188372    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istiod-ca-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:35.188464 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.188379    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kftvd\" (UniqueName: \"kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-kube-api-access-kftvd\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:35.188464 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.188387    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3d116a9-f27c-4e2d-b1aa-ed0453e30551-istio-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:35.192578 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.192549    2577 generic.go:356] "Generic (PLEG): container finished" podID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerID="1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0" exitCode=0
Jul 30 20:27:35.192674 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.192590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" event={"ID":"c3d116a9-f27c-4e2d-b1aa-ed0453e30551","Type":"ContainerDied","Data":"1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0"}
Jul 30 20:27:35.192674 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.192594    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"
Jul 30 20:27:35.192674 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.192638    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z" event={"ID":"c3d116a9-f27c-4e2d-b1aa-ed0453e30551","Type":"ContainerDied","Data":"764b3873ceda0b89fba9bcc786b1efdebe2aeafaeaab075b18717d99e8a29a4f"}
Jul 30 20:27:35.192674 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.192663    2577 scope.go:117] "RemoveContainer" containerID="1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0"
Jul 30 20:27:35.200709 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.200690    2577 scope.go:117] "RemoveContainer" containerID="80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022"
Jul 30 20:27:35.208273 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.208211    2577 scope.go:117] "RemoveContainer" containerID="1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0"
Jul 30 20:27:35.208473 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:27:35.208452    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0\": container with ID starting with 1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0 not found: ID does not exist" containerID="1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0"
Jul 30 20:27:35.208517 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.208479    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0"} err="failed to get container status \"1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0\": rpc error: code = NotFound desc = could not find container \"1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0\": container with ID starting with 1002aba4d168448f6f995bffef12ed903d22347abff0d82d2c0551a51d3ddac0 not found: ID does not exist"
Jul 30 20:27:35.208517 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.208496    2577 scope.go:117] "RemoveContainer" containerID="80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022"
Jul 30 20:27:35.208748 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:27:35.208730    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022\": container with ID starting with 80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022 not found: ID does not exist" containerID="80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022"
Jul 30 20:27:35.208784 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.208755    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022"} err="failed to get container status \"80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022\": rpc error: code = NotFound desc = could not find container \"80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022\": container with ID starting with 80d37fb3243a416437f43959a51ee7066008260353f98ce278266dc8785a3022 not found: ID does not exist"
Jul 30 20:27:35.213047 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.213027    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"]
Jul 30 20:27:35.218613 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:35.218592    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-0c599c-openshift-default-598c956bbd-9sl8z"]
Jul 30 20:27:36.783584 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:36.783550    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" path="/var/lib/kubelet/pods/c3d116a9-f27c-4e2d-b1aa-ed0453e30551/volumes"
Jul 30 20:27:44.061224 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:44.061193    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:27:47.428108 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:47.428060    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"]
Jul 30 20:27:47.428629 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:47.428323    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerName="istio-proxy" containerID="cri-o://90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379" gracePeriod=30
Jul 30 20:27:50.372793 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:50.372761    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 20:27:50.373185 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:50.372813    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 20:27:52.669419 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.669395    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:27:52.715104 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715035    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-socket\") pod \"9abe6e50-80e8-4321-9093-ffee778c4cdf\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") "
Jul 30 20:27:52.715104 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715086    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-credential-socket\") pod \"9abe6e50-80e8-4321-9093-ffee778c4cdf\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") "
Jul 30 20:27:52.715278 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715138    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-data\") pod \"9abe6e50-80e8-4321-9093-ffee778c4cdf\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") "
Jul 30 20:27:52.715278 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715155    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-certs\") pod \"9abe6e50-80e8-4321-9093-ffee778c4cdf\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") "
Jul 30 20:27:52.715278 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715197    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9abe6e50-80e8-4321-9093-ffee778c4cdf-istiod-ca-cert\") pod \"9abe6e50-80e8-4321-9093-ffee778c4cdf\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") "
Jul 30 20:27:52.715278 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715215    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-podinfo\") pod \"9abe6e50-80e8-4321-9093-ffee778c4cdf\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") "
Jul 30 20:27:52.715278 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715232    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-envoy\") pod \"9abe6e50-80e8-4321-9093-ffee778c4cdf\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") "
Jul 30 20:27:52.715278 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715257    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-token\") pod \"9abe6e50-80e8-4321-9093-ffee778c4cdf\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") "
Jul 30 20:27:52.715278 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715250    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9abe6e50-80e8-4321-9093-ffee778c4cdf" (UID: "9abe6e50-80e8-4321-9093-ffee778c4cdf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:27:52.715278 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715275    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7k8fr\" (UniqueName: \"kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-kube-api-access-7k8fr\") pod \"9abe6e50-80e8-4321-9093-ffee778c4cdf\" (UID: \"9abe6e50-80e8-4321-9093-ffee778c4cdf\") "
Jul 30 20:27:52.715686 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715392    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9abe6e50-80e8-4321-9093-ffee778c4cdf" (UID: "9abe6e50-80e8-4321-9093-ffee778c4cdf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:27:52.715686 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715503    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9abe6e50-80e8-4321-9093-ffee778c4cdf" (UID: "9abe6e50-80e8-4321-9093-ffee778c4cdf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:27:52.715686 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715523    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:52.715686 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715546    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-credential-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:52.715887 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.715779    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9abe6e50-80e8-4321-9093-ffee778c4cdf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9abe6e50-80e8-4321-9093-ffee778c4cdf" (UID: "9abe6e50-80e8-4321-9093-ffee778c4cdf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:27:52.716127 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.716105    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-data" (OuterVolumeSpecName: "istio-data") pod "9abe6e50-80e8-4321-9093-ffee778c4cdf" (UID: "9abe6e50-80e8-4321-9093-ffee778c4cdf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:27:52.717454 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.717414    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9abe6e50-80e8-4321-9093-ffee778c4cdf" (UID: "9abe6e50-80e8-4321-9093-ffee778c4cdf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:27:52.717700 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.717530    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-kube-api-access-7k8fr" (OuterVolumeSpecName: "kube-api-access-7k8fr") pod "9abe6e50-80e8-4321-9093-ffee778c4cdf" (UID: "9abe6e50-80e8-4321-9093-ffee778c4cdf"). InnerVolumeSpecName "kube-api-access-7k8fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:27:52.717809 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.717786    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-token" (OuterVolumeSpecName: "istio-token") pod "9abe6e50-80e8-4321-9093-ffee778c4cdf" (UID: "9abe6e50-80e8-4321-9093-ffee778c4cdf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:27:52.717876 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.717858    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9abe6e50-80e8-4321-9093-ffee778c4cdf" (UID: "9abe6e50-80e8-4321-9093-ffee778c4cdf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 20:27:52.816255 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.816235    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k8fr\" (UniqueName: \"kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-kube-api-access-7k8fr\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:52.816255 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.816255    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-data\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:52.816377 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.816265    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-workload-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:52.816377 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.816273    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9abe6e50-80e8-4321-9093-ffee778c4cdf-istiod-ca-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:52.816377 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.816280    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-podinfo\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:52.816377 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.816288    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-envoy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:52.816377 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:52.816297    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9abe6e50-80e8-4321-9093-ffee778c4cdf-istio-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:27:53.274532 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.274505    2577 generic.go:356] "Generic (PLEG): container finished" podID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerID="90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379" exitCode=0
Jul 30 20:27:53.274676 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.274561    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"
Jul 30 20:27:53.274676 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.274576    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" event={"ID":"9abe6e50-80e8-4321-9093-ffee778c4cdf","Type":"ContainerDied","Data":"90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379"}
Jul 30 20:27:53.274676 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.274642    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g" event={"ID":"9abe6e50-80e8-4321-9093-ffee778c4cdf","Type":"ContainerDied","Data":"39064a0dc0e7ccf53e697cffda76f42015b6a0c4d7d3a7e54e10a52d61138a5f"}
Jul 30 20:27:53.274676 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.274668    2577 scope.go:117] "RemoveContainer" containerID="90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379"
Jul 30 20:27:53.282826 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.282804    2577 scope.go:117] "RemoveContainer" containerID="5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee"
Jul 30 20:27:53.290272 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.290255    2577 scope.go:117] "RemoveContainer" containerID="90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379"
Jul 30 20:27:53.290534 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:27:53.290515    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379\": container with ID starting with 90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379 not found: ID does not exist" containerID="90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379"
Jul 30 20:27:53.290623 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.290545    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379"} err="failed to get container status \"90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379\": rpc error: code = NotFound desc = could not find container \"90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379\": container with ID starting with 90b77522dd1e7dbcfc37c1f785e1f48fd6eaf5fe07de8258d3b79bee03e72379 not found: ID does not exist"
Jul 30 20:27:53.290623 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.290568    2577 scope.go:117] "RemoveContainer" containerID="5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee"
Jul 30 20:27:53.290836 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:27:53.290814    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee\": container with ID starting with 5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee not found: ID does not exist" containerID="5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee"
Jul 30 20:27:53.290889 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.290844    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee"} err="failed to get container status \"5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee\": rpc error: code = NotFound desc = could not find container \"5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee\": container with ID starting with 5859f02aab18c9c8e164ec70a026c5bfae0907584df3f8a35532c77bf5892bee not found: ID does not exist"
Jul 30 20:27:53.297194 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.297174    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"]
Jul 30 20:27:53.300890 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:53.300872    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-d67d52-openshift-default-595cc4d76-4p49g"]
Jul 30 20:27:54.783922 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:27:54.783887    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" path="/var/lib/kubelet/pods/9abe6e50-80e8-4321-9093-ffee778c4cdf/volumes"
Jul 30 20:28:01.561955 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:01.561924    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:28:15.734061 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:15.734024    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"]
Jul 30 20:28:15.734554 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:15.734285    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw" podUID="ce5b8960-f89b-40f2-8761-3e45de27fbbb" containerName="maas-api" containerID="cri-o://3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9" gracePeriod=30
Jul 30 20:28:15.971221 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:15.971196    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:28:16.074701 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.074672    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce5b8960-f89b-40f2-8761-3e45de27fbbb-openshift-service-ca\") pod \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") "
Jul 30 20:28:16.074853 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.074732    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qnr2\" (UniqueName: \"kubernetes.io/projected/ce5b8960-f89b-40f2-8761-3e45de27fbbb-kube-api-access-6qnr2\") pod \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") "
Jul 30 20:28:16.074853 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.074822    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce5b8960-f89b-40f2-8761-3e45de27fbbb-maas-api-tls\") pod \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\" (UID: \"ce5b8960-f89b-40f2-8761-3e45de27fbbb\") "
Jul 30 20:28:16.075121 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.075059    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce5b8960-f89b-40f2-8761-3e45de27fbbb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce5b8960-f89b-40f2-8761-3e45de27fbbb" (UID: "ce5b8960-f89b-40f2-8761-3e45de27fbbb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:28:16.076774 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.076752    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce5b8960-f89b-40f2-8761-3e45de27fbbb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce5b8960-f89b-40f2-8761-3e45de27fbbb" (UID: "ce5b8960-f89b-40f2-8761-3e45de27fbbb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:28:16.076840 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.076792    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce5b8960-f89b-40f2-8761-3e45de27fbbb-kube-api-access-6qnr2" (OuterVolumeSpecName: "kube-api-access-6qnr2") pod "ce5b8960-f89b-40f2-8761-3e45de27fbbb" (UID: "ce5b8960-f89b-40f2-8761-3e45de27fbbb"). InnerVolumeSpecName "kube-api-access-6qnr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:28:16.175693 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.175662    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce5b8960-f89b-40f2-8761-3e45de27fbbb-maas-api-tls\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:16.175693 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.175688    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce5b8960-f89b-40f2-8761-3e45de27fbbb-openshift-service-ca\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:16.175693 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.175697    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qnr2\" (UniqueName: \"kubernetes.io/projected/ce5b8960-f89b-40f2-8761-3e45de27fbbb-kube-api-access-6qnr2\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:16.365630 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.365548    2577 generic.go:356] "Generic (PLEG): container finished" podID="ce5b8960-f89b-40f2-8761-3e45de27fbbb" containerID="3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9" exitCode=0
Jul 30 20:28:16.365630 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.365605    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"
Jul 30 20:28:16.365826 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.365608    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw" event={"ID":"ce5b8960-f89b-40f2-8761-3e45de27fbbb","Type":"ContainerDied","Data":"3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9"}
Jul 30 20:28:16.365826 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.365712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw" event={"ID":"ce5b8960-f89b-40f2-8761-3e45de27fbbb","Type":"ContainerDied","Data":"0a7e8b48ab3ed8cd37b02b243acd0459a6fbcab564037e2e7bac7be396dbdbce"}
Jul 30 20:28:16.365826 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.365744    2577 scope.go:117] "RemoveContainer" containerID="3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9"
Jul 30 20:28:16.373873 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.373851    2577 scope.go:117] "RemoveContainer" containerID="3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9"
Jul 30 20:28:16.374192 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:28:16.374174    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9\": container with ID starting with 3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9 not found: ID does not exist" containerID="3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9"
Jul 30 20:28:16.374263 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.374200    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9"} err="failed to get container status \"3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9\": rpc error: code = NotFound desc = could not find container \"3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9\": container with ID starting with 3d25d7e646f883dd97d08108badb1f873e2c8f4b0fef2058525dc80cbb47c8d9 not found: ID does not exist"
Jul 30 20:28:16.385923 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.385904    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"]
Jul 30 20:28:16.390081 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.390045    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a769e5-57fc75ccbd-7lfrw"]
Jul 30 20:28:16.785553 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:16.785481    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce5b8960-f89b-40f2-8761-3e45de27fbbb" path="/var/lib/kubelet/pods/ce5b8960-f89b-40f2-8761-3e45de27fbbb/volumes"
Jul 30 20:28:18.859843 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:18.859806    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gkllt"]
Jul 30 20:28:22.619438 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:22.619407    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"]
Jul 30 20:28:22.619894 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:22.619604    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy" containerID="cri-o://8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d" gracePeriod=30
Jul 30 20:28:23.293378 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:23.293342    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8d8b8fb6f-qtrd8_9593661e-3eda-4fc5-a1f4-52da8cb9d919/authorino/0.log"
Jul 30 20:28:27.861587 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.861560    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:28:27.951811 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.951740    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-envoy\") pod \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") "
Jul 30 20:28:27.951811 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.951779    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-data\") pod \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") "
Jul 30 20:28:27.952002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.951816    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-certs\") pod \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") "
Jul 30 20:28:27.952002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.951873    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istiod-ca-cert\") pod \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") "
Jul 30 20:28:27.952002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.951902    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-credential-socket\") pod \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") "
Jul 30 20:28:27.952193 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952104    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8g76\" (UniqueName: \"kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-kube-api-access-t8g76\") pod \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") "
Jul 30 20:28:27.952193 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952160    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-podinfo\") pod \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") "
Jul 30 20:28:27.952193 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952191    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-token\") pod \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") "
Jul 30 20:28:27.952344 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952181    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5ef11c24-cd63-40d6-8ac7-872c3a6858dd" (UID: "5ef11c24-cd63-40d6-8ac7-872c3a6858dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:28:27.952344 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952204    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5ef11c24-cd63-40d6-8ac7-872c3a6858dd" (UID: "5ef11c24-cd63-40d6-8ac7-872c3a6858dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:28:27.952344 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5ef11c24-cd63-40d6-8ac7-872c3a6858dd" (UID: "5ef11c24-cd63-40d6-8ac7-872c3a6858dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:28:27.952344 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952316    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-socket\") pod \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\" (UID: \"5ef11c24-cd63-40d6-8ac7-872c3a6858dd\") "
Jul 30 20:28:27.952551 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952535    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:27.952604 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952555    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istiod-ca-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:27.952604 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952569    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-credential-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:27.952749 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952678    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "5ef11c24-cd63-40d6-8ac7-872c3a6858dd" (UID: "5ef11c24-cd63-40d6-8ac7-872c3a6858dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:28:27.952819 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.952792    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5ef11c24-cd63-40d6-8ac7-872c3a6858dd" (UID: "5ef11c24-cd63-40d6-8ac7-872c3a6858dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:28:27.954292 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.954267    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-kube-api-access-t8g76" (OuterVolumeSpecName: "kube-api-access-t8g76") pod "5ef11c24-cd63-40d6-8ac7-872c3a6858dd" (UID: "5ef11c24-cd63-40d6-8ac7-872c3a6858dd"). InnerVolumeSpecName "kube-api-access-t8g76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:28:27.954400 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.954289    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5ef11c24-cd63-40d6-8ac7-872c3a6858dd" (UID: "5ef11c24-cd63-40d6-8ac7-872c3a6858dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 20:28:27.954400 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.954292    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5ef11c24-cd63-40d6-8ac7-872c3a6858dd" (UID: "5ef11c24-cd63-40d6-8ac7-872c3a6858dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:28:27.954400 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:27.954381    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "5ef11c24-cd63-40d6-8ac7-872c3a6858dd" (UID: "5ef11c24-cd63-40d6-8ac7-872c3a6858dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:28:28.053726 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.053705    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-data\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:28.053726 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.053724    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8g76\" (UniqueName: \"kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-kube-api-access-t8g76\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:28.053852 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.053735    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-podinfo\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:28.053852 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.053745    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:28.053852 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.053753    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-workload-socket\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:28.053852 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.053761    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ef11c24-cd63-40d6-8ac7-872c3a6858dd-istio-envoy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\""
Jul 30 20:28:28.414579 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.414554    2577 generic.go:356] "Generic (PLEG): container finished" podID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerID="8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d" exitCode=0
Jul 30 20:28:28.414678 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.414602    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"
Jul 30 20:28:28.414678 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.414640    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" event={"ID":"5ef11c24-cd63-40d6-8ac7-872c3a6858dd","Type":"ContainerDied","Data":"8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d"}
Jul 30 20:28:28.414755 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.414693    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm" event={"ID":"5ef11c24-cd63-40d6-8ac7-872c3a6858dd","Type":"ContainerDied","Data":"c3c57edb9f5e409816d666a8e394cf51c7e1bfd4d1d5a047b3381fb49c1afa83"}
Jul 30 20:28:28.414755 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.414725    2577 scope.go:117] "RemoveContainer" containerID="8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d"
Jul 30 20:28:28.422886 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.422688    2577 scope.go:117] "RemoveContainer" containerID="00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1"
Jul 30 20:28:28.430163 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.430147    2577 scope.go:117] "RemoveContainer" containerID="8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d"
Jul 30 20:28:28.430407 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:28:28.430389    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d\": container with ID starting with 8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d not found: ID does not exist" containerID="8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d"
Jul 30 20:28:28.430455 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.430415    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d"} err="failed to get container status \"8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d\": rpc error: code = NotFound desc = could not find container \"8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d\": container with ID starting with 8426008ea82c26d4708d7475f1278fa24dbb7f8cab65f361cc3374316c9f021d not found: ID does not exist"
Jul 30 20:28:28.430455 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.430430    2577 scope.go:117] "RemoveContainer" containerID="00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1"
Jul 30 20:28:28.430628 ip-10-0-129-203 kubenswrapper[2577]: E0730 20:28:28.430612    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1\": container with ID starting with 00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1 not found: ID does not exist" containerID="00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1"
Jul 30 20:28:28.430664 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.430631    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1"} err="failed to get container status \"00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1\": rpc error: code = NotFound desc = could not find container \"00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1\": container with ID starting with 00e73956717cb39d274eaf48f8fcb595c9bcafc70eee7c3c4ebcf86a6a707fd1 not found: ID does not exist"
Jul 30 20:28:28.435962 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.435940    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"]
Jul 30 20:28:28.439590 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.439573    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-a769e5-openshift-default-69647f999b-qbjvm"]
Jul 30 20:28:28.788218 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:28.788149    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" path="/var/lib/kubelet/pods/5ef11c24-cd63-40d6-8ac7-872c3a6858dd/volumes"
Jul 30 20:28:29.233429 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:29.233404    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6_647aaeb7-63c8-4ffc-ab6e-ea808a667b72/manager/0.log"
Jul 30 20:28:29.240187 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:29.240164    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6_647aaeb7-63c8-4ffc-ab6e-ea808a667b72/copy-manifests/0.log"
Jul 30 20:28:31.578287 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:31.578262    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8d8b8fb6f-qtrd8_9593661e-3eda-4fc5-a1f4-52da8cb9d919/authorino/0.log"
Jul 30 20:28:32.274595 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:32.274566    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-gkllt_9d6b89b3-e211-4e72-8069-4c3931d99f21/limitador/0.log"
Jul 30 20:28:33.086108 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:33.086060    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4b6cc975-kp2nh_05ea1da3-9e19-42a4-aa1f-a6de6f74c027/kube-auth-proxy/0.log"
Jul 30 20:28:34.485141 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:34.485112    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-jtws7_98667ffa-8b3d-4f77-bdf1-82a6bf75177c/main/0.log"
Jul 30 20:28:39.880535 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880496    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fmzs6/must-gather-5p8kc"]
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880880    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy"
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880899    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy"
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880911    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy"
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880920    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy"
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880931    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerName="istio-proxy"
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880940    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerName="istio-proxy"
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880952    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" containerName="main"
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880959    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" containerName="main"
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880976    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="663d5460-ac42-4131-b9d4-c43c80bdf6de" containerName="maas-api"
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880983    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="663d5460-ac42-4131-b9d4-c43c80bdf6de" containerName="maas-api"
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.880996    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce5b8960-f89b-40f2-8761-3e45de27fbbb" containerName="maas-api"
Jul 30 20:28:39.881002 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881003    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce5b8960-f89b-40f2-8761-3e45de27fbbb" containerName="maas-api"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881014    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881022    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881031    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881039    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881136    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881147    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerName="istio-proxy"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881155    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f49f8aa1-cdbb-403a-b578-20ad89bd3ed4" containerName="main"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881163    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881172    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="663d5460-ac42-4131-b9d4-c43c80bdf6de" containerName="maas-api"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881180    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerName="istio-proxy"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881188    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ef11c24-cd63-40d6-8ac7-872c3a6858dd" containerName="istio-proxy"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881198    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce5b8960-f89b-40f2-8761-3e45de27fbbb" containerName="maas-api"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881276    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerName="istio-proxy"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881286    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9abe6e50-80e8-4321-9093-ffee778c4cdf" containerName="istio-proxy"
Jul 30 20:28:39.881625 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.881350    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d116a9-f27c-4e2d-b1aa-ed0453e30551" containerName="istio-proxy"
Jul 30 20:28:39.885806 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.885784    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fmzs6/must-gather-5p8kc"
Jul 30 20:28:39.888485 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.888462    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fmzs6\"/\"openshift-service-ca.crt\""
Jul 30 20:28:39.888585 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.888463    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fmzs6\"/\"kube-root-ca.crt\""
Jul 30 20:28:39.889779 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.889758    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fmzs6\"/\"default-dockercfg-xc2gz\""
Jul 30 20:28:39.891328 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:39.891305    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fmzs6/must-gather-5p8kc"]
Jul 30 20:28:40.030792 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:40.030763    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/32b20af5-8149-408d-9c8f-fcb1e16531d2-must-gather-output\") pod \"must-gather-5p8kc\" (UID: \"32b20af5-8149-408d-9c8f-fcb1e16531d2\") " pod="openshift-must-gather-fmzs6/must-gather-5p8kc"
Jul 30 20:28:40.030792 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:40.030793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-525r7\" (UniqueName: \"kubernetes.io/projected/32b20af5-8149-408d-9c8f-fcb1e16531d2-kube-api-access-525r7\") pod \"must-gather-5p8kc\" (UID: \"32b20af5-8149-408d-9c8f-fcb1e16531d2\") " pod="openshift-must-gather-fmzs6/must-gather-5p8kc"
Jul 30 20:28:40.131657 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:40.131603    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/32b20af5-8149-408d-9c8f-fcb1e16531d2-must-gather-output\") pod \"must-gather-5p8kc\" (UID: \"32b20af5-8149-408d-9c8f-fcb1e16531d2\") " pod="openshift-must-gather-fmzs6/must-gather-5p8kc"
Jul 30 20:28:40.131657 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:40.131627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-525r7\" (UniqueName: \"kubernetes.io/projected/32b20af5-8149-408d-9c8f-fcb1e16531d2-kube-api-access-525r7\") pod \"must-gather-5p8kc\" (UID: \"32b20af5-8149-408d-9c8f-fcb1e16531d2\") " pod="openshift-must-gather-fmzs6/must-gather-5p8kc"
Jul 30 20:28:40.131901 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:40.131884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/32b20af5-8149-408d-9c8f-fcb1e16531d2-must-gather-output\") pod \"must-gather-5p8kc\" (UID: \"32b20af5-8149-408d-9c8f-fcb1e16531d2\") " pod="openshift-must-gather-fmzs6/must-gather-5p8kc"
Jul 30 20:28:40.139270 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:40.139239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-525r7\" (UniqueName: \"kubernetes.io/projected/32b20af5-8149-408d-9c8f-fcb1e16531d2-kube-api-access-525r7\") pod \"must-gather-5p8kc\" (UID: \"32b20af5-8149-408d-9c8f-fcb1e16531d2\") " pod="openshift-must-gather-fmzs6/must-gather-5p8kc"
Jul 30 20:28:40.195636 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:40.195612    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fmzs6/must-gather-5p8kc"
Jul 30 20:28:40.324184 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:40.324154    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fmzs6/must-gather-5p8kc"]
Jul 30 20:28:40.326988 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:28:40.326956    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32b20af5_8149_408d_9c8f_fcb1e16531d2.slice/crio-46963ffd297e59e9ae3bee532319e0c4a4c44ff54d89d6433b30eefe28faf9be WatchSource:0}: Error finding container 46963ffd297e59e9ae3bee532319e0c4a4c44ff54d89d6433b30eefe28faf9be: Status 404 returned error can't find the container with id 46963ffd297e59e9ae3bee532319e0c4a4c44ff54d89d6433b30eefe28faf9be
Jul 30 20:28:40.328739 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:40.328721    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:28:40.462043 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:40.461983    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fmzs6/must-gather-5p8kc" event={"ID":"32b20af5-8149-408d-9c8f-fcb1e16531d2","Type":"ContainerStarted","Data":"46963ffd297e59e9ae3bee532319e0c4a4c44ff54d89d6433b30eefe28faf9be"}
Jul 30 20:28:41.470521 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:41.470478    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fmzs6/must-gather-5p8kc" event={"ID":"32b20af5-8149-408d-9c8f-fcb1e16531d2","Type":"ContainerStarted","Data":"9be0e949b8ffe07e949d4ddbbb16f80dc8b9097f9b185dc29a9073490b93487c"}
Jul 30 20:28:41.470521 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:41.470527    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fmzs6/must-gather-5p8kc" event={"ID":"32b20af5-8149-408d-9c8f-fcb1e16531d2","Type":"ContainerStarted","Data":"23855f881c0d483ddf7801c39fc14dd56199739df9170f79deb13785194dfb07"}
Jul 30 20:28:41.488190 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:41.488136    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fmzs6/must-gather-5p8kc" podStartSLOduration=1.637513297 podStartE2EDuration="2.488118379s" totalImagesPullingTime="850.605082ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:28:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:28:41.487727005 +0000 UTC m=+3841.248482264" watchObservedRunningTime="2026-07-30 20:28:41.488118379 +0000 UTC m=+3841.248873637"
Jul 30 20:28:42.678155 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:42.678124    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7dmqn_3fe769f8-bc2c-466c-be78-95ef7f74263c/global-pull-secret-syncer/0.log"
Jul 30 20:28:42.802620 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:42.802587    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p7ltn_15de2c06-37ea-4122-9eaf-5fbe481c3127/konnectivity-agent/0.log"
Jul 30 20:28:42.874252 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:42.874222    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-203.ec2.internal_c7684a39bc1278f7d06d66597af84743/haproxy/0.log"
Jul 30 20:28:48.029119 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:48.029065    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8d8b8fb6f-qtrd8_9593661e-3eda-4fc5-a1f4-52da8cb9d919/authorino/0.log"
Jul 30 20:28:48.296953 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:48.296869    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-gkllt_9d6b89b3-e211-4e72-8069-4c3931d99f21/limitador/0.log"
Jul 30 20:28:50.343461 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:50.343380    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6f8cs_7b76526d-44ac-43f8-acfa-b1feb0ea88ea/node-exporter/0.log"
Jul 30 20:28:50.432289 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:50.432230    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6f8cs_7b76526d-44ac-43f8-acfa-b1feb0ea88ea/kube-rbac-proxy/0.log"
Jul 30 20:28:50.472333 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:50.472276    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6f8cs_7b76526d-44ac-43f8-acfa-b1feb0ea88ea/init-textfile/0.log"
Jul 30 20:28:52.828972 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.828923    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"]
Jul 30 20:28:52.834924 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.834891    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.843887 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.843838    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"]
Jul 30 20:28:52.857734 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.857281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-lib-modules\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.857734 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.857334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc4pl\" (UniqueName: \"kubernetes.io/projected/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-kube-api-access-nc4pl\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.857734 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.857373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-proc\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.857734 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.857406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-sys\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.857734 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.857515    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-podres\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.958454 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.958410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-podres\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.958649 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.958467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-lib-modules\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.958649 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.958505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc4pl\" (UniqueName: \"kubernetes.io/projected/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-kube-api-access-nc4pl\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.958649 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.958543    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-proc\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.958649 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.958573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-sys\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.958894 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.958697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-sys\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.958894 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.958809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-podres\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.959008 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.958903    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-lib-modules\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.959333 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.959304    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-proc\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:52.967529 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:52.967499    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc4pl\" (UniqueName: \"kubernetes.io/projected/6c1b0f49-7a90-4e90-bf23-fbee0ef319a5-kube-api-access-nc4pl\") pod \"perf-node-gather-daemonset-x6pld\" (UID: \"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:53.152464 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:53.152383    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:53.339101 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:53.339045    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"]
Jul 30 20:28:53.342742 ip-10-0-129-203 kubenswrapper[2577]: W0730 20:28:53.342704    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6c1b0f49_7a90_4e90_bf23_fbee0ef319a5.slice/crio-d91d32e0cb225cf1fc1eec0f3d9558bf15f50eeec3c94ba360fcd0c13a338423 WatchSource:0}: Error finding container d91d32e0cb225cf1fc1eec0f3d9558bf15f50eeec3c94ba360fcd0c13a338423: Status 404 returned error can't find the container with id d91d32e0cb225cf1fc1eec0f3d9558bf15f50eeec3c94ba360fcd0c13a338423
Jul 30 20:28:53.549320 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:53.549278    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld" event={"ID":"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5","Type":"ContainerStarted","Data":"777e31164436ac53caa74bc729aa07bd23f4460dd63e7046fae7e5ec28acc2d7"}
Jul 30 20:28:53.549526 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:53.549345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld" event={"ID":"6c1b0f49-7a90-4e90-bf23-fbee0ef319a5","Type":"ContainerStarted","Data":"d91d32e0cb225cf1fc1eec0f3d9558bf15f50eeec3c94ba360fcd0c13a338423"}
Jul 30 20:28:53.550089 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:53.550049    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:28:53.567841 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:53.567774    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld" podStartSLOduration=1.567756749 podStartE2EDuration="1.567756749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:28:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:28:53.565873648 +0000 UTC m=+3853.326628912" watchObservedRunningTime="2026-07-30 20:28:53.567756749 +0000 UTC m=+3853.328512005"
Jul 30 20:28:55.155464 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:55.155420    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7mqgt_c153fb6e-27af-4e8e-a363-1bc0922c458e/dns/0.log"
Jul 30 20:28:55.179546 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:55.179518    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7mqgt_c153fb6e-27af-4e8e-a363-1bc0922c458e/kube-rbac-proxy/0.log"
Jul 30 20:28:55.288725 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:55.288699    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4b287_276fa5f2-21c9-42fc-88fc-6f7d3db559c9/dns-node-resolver/0.log"
Jul 30 20:28:55.841940 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:55.841912    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7j96x_3a3a6e9b-c842-4a6d-a79f-49efa6cf02a1/node-ca/0.log"
Jul 30 20:28:57.057324 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:57.057295    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4b6cc975-kp2nh_05ea1da3-9e19-42a4-aa1f-a6de6f74c027/kube-auth-proxy/0.log"
Jul 30 20:28:57.971283 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:57.971253    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ftznv_a958f2b4-3331-4143-8794-7297a6e533ec/serve-healthcheck-canary/0.log"
Jul 30 20:28:58.602541 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:58.602471    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s47sv_288e7048-1964-4683-ab92-3fcacea15656/kube-rbac-proxy/0.log"
Jul 30 20:28:58.622494 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:58.622461    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s47sv_288e7048-1964-4683-ab92-3fcacea15656/exporter/0.log"
Jul 30 20:28:58.642774 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:28:58.642750    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s47sv_288e7048-1964-4683-ab92-3fcacea15656/extractor/0.log"
Jul 30 20:29:00.571690 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:00.571658    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-x6pld"
Jul 30 20:29:01.164484 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:01.164444    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6_647aaeb7-63c8-4ffc-ab6e-ea808a667b72/manager/0.log"
Jul 30 20:29:01.187123 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:01.187020    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdfbdbf4-gbjs6_647aaeb7-63c8-4ffc-ab6e-ea808a667b72/copy-manifests/0.log"
Jul 30 20:29:02.736563 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:02.736529    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-659fd749fb-6fq69_0e086d1f-d826-4bd4-bb39-2930137f9d87/manager/0.log"
Jul 30 20:29:03.118087 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:03.118043    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4b6cc975-kp2nh_05ea1da3-9e19-42a4-aa1f-a6de6f74c027/kube-auth-proxy/0.log"
Jul 30 20:29:04.782435 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:04.782361    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4b6cc975-kp2nh_05ea1da3-9e19-42a4-aa1f-a6de6f74c027/kube-auth-proxy/0.log"
Jul 30 20:29:07.543631 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:07.543602    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-76dtc_1eee9ece-e46a-432a-aced-5004c9ed090b/migrator/0.log"
Jul 30 20:29:07.566541 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:07.566515    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-76dtc_1eee9ece-e46a-432a-aced-5004c9ed090b/graceful-termination/0.log"
Jul 30 20:29:09.108147 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:09.108119    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9b6xj_7b945f77-954b-47f9-a8db-8c6c586dc346/kube-multus-additional-cni-plugins/0.log"
Jul 30 20:29:09.135247 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:09.135222    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9b6xj_7b945f77-954b-47f9-a8db-8c6c586dc346/egress-router-binary-copy/0.log"
Jul 30 20:29:09.158036 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:09.158015    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9b6xj_7b945f77-954b-47f9-a8db-8c6c586dc346/cni-plugins/0.log"
Jul 30 20:29:09.183976 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:09.183953    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9b6xj_7b945f77-954b-47f9-a8db-8c6c586dc346/bond-cni-plugin/0.log"
Jul 30 20:29:09.206587 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:09.206568    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9b6xj_7b945f77-954b-47f9-a8db-8c6c586dc346/routeoverride-cni/0.log"
Jul 30 20:29:09.232903 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:09.232881    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9b6xj_7b945f77-954b-47f9-a8db-8c6c586dc346/whereabouts-cni-bincopy/0.log"
Jul 30 20:29:09.255262 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:09.255237    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9b6xj_7b945f77-954b-47f9-a8db-8c6c586dc346/whereabouts-cni/0.log"
Jul 30 20:29:09.500524 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:09.500465    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tgv2k_0880f05e-770d-46dd-a957-08c121ec9fac/kube-multus/0.log"
Jul 30 20:29:09.518792 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:09.518769    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-26gjf_99fc519b-8e3c-4564-a65a-f81a314974f9/network-metrics-daemon/0.log"
Jul 30 20:29:09.535635 ip-10-0-129-203 kubenswrapper[2577]: I0730 20:29:09.535601    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-26gjf_99fc519b-8e3c-4564-a65a-f81a314974f9/kube-rbac-proxy/0.log"