Jul 24 14:32:01.377398 ip-10-0-143-246 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.780698    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784341    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784353    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784356    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784359    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784363    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784366    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784371    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:32:01.867622 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784374    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784377    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784379    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784382    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784385    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784388    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784390    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784395    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784399    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784402    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784404    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784407    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784409    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784412    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784416    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784419    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784421    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784426    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784429    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:32:01.871935 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784433    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784436    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784438    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784441    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784443    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784446    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784448    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784451    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784454    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784456    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784459    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784462    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784464    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784467    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784470    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784472    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784475    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784478    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784481    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:32:01.872680 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784484    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784487    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784489    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784492    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784495    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784498    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784502    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784505    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784508    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784511    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784514    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784516    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784519    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784521    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784524    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784527    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784529    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784532    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784535    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:32:01.882478 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784537    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784540    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784543    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784545    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784548    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784550    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784553    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784556    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784559    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784562    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784565    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784568    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784570    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784573    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784576    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784578    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784581    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784584    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784586    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784589    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:32:01.883206 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784591    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784595    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784598    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784600    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784604    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784607    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784609    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784612    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784614    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784617    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784620    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784622    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784625    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784628    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.784630    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785337    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785344    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785347    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785350    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785353    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:32:01.883896 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785357    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785360    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785363    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785366    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785368    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785371    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785374    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785376    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785379    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785382    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785385    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785388    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785390    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785393    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785395    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785398    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785401    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785404    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785407    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785409    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:32:01.884633 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785412    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785414    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785417    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785419    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785422    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785424    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785427    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785430    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785432    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785435    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785437    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785440    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785442    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785445    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785448    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785450    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785453    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785455    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785457    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:32:01.885358 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785462    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785466    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785469    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785472    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785474    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785477    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785480    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785486    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785489    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785492    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785495    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785498    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785501    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785504    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785508    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785511    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785514    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785517    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785520    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:32:01.886079 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785522    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785525    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785528    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785531    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785533    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785536    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785538    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785541    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785544    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785547    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785549    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785552    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785555    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785558    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785560    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785563    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785566    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785569    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785571    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785574    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:32:01.886781 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785577    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785580    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785582    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785585    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785587    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785590    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785593    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785595    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785598    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785600    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785604    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785607    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785610    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785612    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785615    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785621    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.785998    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786004    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786010    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:32:01.977914 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786014    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786017    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786020    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786023    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786026    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786030    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786032    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786035    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786038    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786041    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786044    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786046    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786049    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786051    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786054    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786056    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786059    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786061    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786064    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:32:01.979672 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786069    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786072    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786075    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786077    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786080    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786082    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786085    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786088    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786105    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786108    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786111    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786115    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786120    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786123    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786125    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786129    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786132    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786135    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786137    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786140    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:32:01.980440 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786142    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786145    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786147    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786150    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786153    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786155    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786157    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786160    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786163    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786165    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786168    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786171    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786175    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786177    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786180    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786184    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786186    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786189    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786192    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786194    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:32:01.981739 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786197    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786199    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786202    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786205    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786207    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786211    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786214    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786217    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786220    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786224    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786227    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786229    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786232    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786234    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786237    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786240    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786242    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786244    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786247    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786250    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:32:01.982473 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786252    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786255    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786257    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786259    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786263    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786266    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786268    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786271    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786273    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786276    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786278    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786280    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786283    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786285    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786288    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786290    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.786293    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786362    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786372    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786381    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786389    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 14:32:01.983258 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786400    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786406    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786411    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786415    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786418    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786422    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786425    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786428    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786431    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786434    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786437    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786440    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786443    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786446    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786450    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786454    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786458    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786461    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786464    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786468    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786471    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786474    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786477    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786480    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 14:32:01.983977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786483    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786486    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786489    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786493    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786497    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786500    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786503    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786506    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786510    2581 flags.go:64] FLAG: --enable-server="true"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786513    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786518    2581 flags.go:64] FLAG: --event-burst="100"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786521    2581 flags.go:64] FLAG: --event-qps="50"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786524    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786527    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786530    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786533    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786536    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786539    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786541    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786544    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786547    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786550    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786553    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786557    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786560    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 14:32:01.985109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786563    2581 flags.go:64] FLAG: --feature-gates=""
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786567    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786570    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786573    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786576    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786579    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786582    2581 flags.go:64] FLAG: --help="false"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786585    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-143-246.ec2.internal"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786588    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786591    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786594    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786597    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786601    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786604    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786607    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786611    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786614    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786618    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786621    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786623    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786626    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786629    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786632    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786635    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 14:32:01.986255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786638    2581 flags.go:64] FLAG: --lock-file=""
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786641    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786644    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786647    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786652    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786655    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786659    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786662    2581 flags.go:64] FLAG: --logging-format="text"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786665    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786668    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786671    2581 flags.go:64] FLAG: --manifest-url=""
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786674    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786679    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786682    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786686    2581 flags.go:64] FLAG: --max-pods="110"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786689    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786692    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786695    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786697    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786701    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786704    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786706    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786714    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786719    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786722    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 14:32:01.987404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786725    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786728    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786734    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786737    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786740    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786743    2581 flags.go:64] FLAG: --port="10250"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786746    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786749    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-069a7f15bc1da2461"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786752    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786755    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786758    2581 flags.go:64] FLAG: --register-node="true"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786761    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786764    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786768    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786771    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786774    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786778    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786781    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786784    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786787    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786790    2581 flags.go:64] FLAG: --runonce="false"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786793    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786796    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786799    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786802    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 14:32:01.988562 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786804    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786807    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786811    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786814    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786817    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786820    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786825    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786827    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786830    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786834    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786837    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786842    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786845    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786848    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786852    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786856    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786858    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786861    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786864    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786868    2581 flags.go:64] FLAG: --v="2"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786873    2581 flags.go:64] FLAG: --version="false"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786877    2581 flags.go:64] FLAG: --vmodule=""
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786881    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.786884    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787003    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:32:01.989694 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787007    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787010    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787013    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787017    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787020    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787023    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787025    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787028    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787031    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787033    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787036    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787039    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787042    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787047    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787050    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787053    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787055    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787064    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787067    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787069    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:32:01.990804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787072    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787074    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787077    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787080    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787082    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787085    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787087    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787104    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787106    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787109    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787111    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787114    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787117    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787119    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787122    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787124    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787127    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787129    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787132    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787134    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:32:01.995064 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787137    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787140    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787144    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787148    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787151    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787155    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787159    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787162    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787165    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787168    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787177    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787180    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787183    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787185    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787189    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787193    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787196    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787198    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787201    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:32:01.995804 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787205    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787207    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787210    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787212    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787215    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787217    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787220    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787222    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787225    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787227    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787230    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787233    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787235    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787238    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787240    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787243    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787245    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787248    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787252    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787254    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:32:01.997354 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787257    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787259    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787262    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787264    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787267    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787270    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787273    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787275    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787278    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787280    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787283    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787285    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787289    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787292    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787294    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787297    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787299    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787302    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:32:01.998099 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.787304    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.788083    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.795107    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.795123    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.795142    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.795151    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795213    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795218    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795222    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795225    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795228    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795232    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795235    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795238    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795240    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:32:01.999236 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795243    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795246    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795250    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795254    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795258    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795263    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795268    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795271    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795273    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795276    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795278    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795281    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795284    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795286    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795289    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795291    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795294    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795296    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795299    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795301    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:32:01.999986 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795305    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795308    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795314    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795317    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795321    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795325    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795328    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795332    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795336    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795341    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795345    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795349    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795352    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795355    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795358    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795361    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795363    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795366    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795369    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:32:02.098962 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795372    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795375    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795377    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795379    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795382    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795385    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795387    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795390    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795393    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795396    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795399    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795401    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795405    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795408    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795413    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795419    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795424    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795428    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795431    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:32:02.099718 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795434    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795437    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795439    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795442    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795444    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795447    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795449    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795452    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795455    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795457    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795460    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795462    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795465    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795467    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795470    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795473    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795475    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795477    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795480    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795482    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:32:02.101227 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795485    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795487    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795490    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795494    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795500    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795505    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795509    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795512    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795516    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795519    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795521    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795524    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.795529    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795696    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795702    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:32:02.101977 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795705    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795708    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795710    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795713    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795716    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795720    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795724    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795726    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795729    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795732    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795735    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795739    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795743    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795747    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795751    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795755    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795758    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795761    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795763    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:32:02.103120 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795767    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795771    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795774    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795779    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795782    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795785    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795790    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795795    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795798    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795800    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795802    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795805    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795808    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795810    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795813    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795815    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795819    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795823    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795827    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:32:02.109315 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795831    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795835    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795839    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795841    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795844    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795846    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795849    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795851    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795854    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795856    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795858    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795861    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795863    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795866    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795869    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795871    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795875    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795878    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795880    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795884    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:32:02.110060 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795886    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795892    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795895    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795897    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795901    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795905    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795910    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795914    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795918    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795921    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795923    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795926    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795929    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795931    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795934    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795936    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795939    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795941    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795944    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795946    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:32:02.110812 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795948    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795951    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795954    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795956    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795959    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795961    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795964    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795966    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795970    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795972    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795975    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795979    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795982    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795986    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795991    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.795996    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.796000    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.796003    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:32:02.112725 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:01.796005    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:32:02.113481 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.796010    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 14:32:02.113481 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.796135    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 14:32:02.113481 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.798383    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 14:32:02.113481 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.799771    2581 server.go:1019] "Starting client certificate rotation"
Jul 24 14:32:02.113481 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.799877    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 14:32:02.113481 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.800525    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 14:32:02.113481 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.802036    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 14:32:02.113481 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:01.808312    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 14:32:03.130221 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.130196    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 24 14:32:03.132701 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.132679    2581 log.go:25] "Validated CRI v1 image API"
Jul 24 14:32:03.133785 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.133763    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 14:32:03.139638 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.139616    2581 fs.go:135] Filesystem UUIDs: map[243bf3fe-78f9-432b-8062-88d414f2a1a5:/dev/nvme0n1p3 6616b0e1-572e-4f88-8dc4-4892b120f666:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 24 14:32:03.139724 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.139637    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 14:32:03.140565 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.140544    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 14:32:03.145696 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.145577    2581 manager.go:216] Machine: {Timestamp:2026-07-24 14:32:03.144083883 +0000 UTC m=+1.647801838 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098410 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec287bae428202014d60e9abe92252cb SystemUUID:ec287bae-4282-0201-4d60-e9abe92252cb BootID:7a24b8d9-2b13-4fce-b85a-073babdd8809 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:92:af:93:40:a9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:92:af:93:40:a9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:ff:b6:3c:b0:5a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 14:32:03.145696 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.145692    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 14:32:03.145825 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.145781    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 14:32:03.148896 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.148865    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 14:32:03.149051 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.148900    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-246.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 14:32:03.150359 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.150348    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 14:32:03.150395 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.150362    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 14:32:03.150395 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.150370    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 14:32:03.151701 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.151691    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 14:32:03.152623 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.152612    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 14:32:03.154831 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.154819    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 14:32:03.154964 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.154953    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 14:32:03.155007 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.154987    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 14:32:03.155007 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.154998    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 14:32:03.155065 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.155013    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 14:32:03.155065 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.155023    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 24 14:32:03.155065 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.155037    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 14:32:03.155464 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.155451    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 14:32:03.155524 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.155473    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 14:32:03.158165 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.158144    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 14:32:03.160545 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.160503    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 14:32:03.160597 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.160564    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 14:32:03.162471 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162459    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 14:32:03.162509 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162478    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 14:32:03.162509 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162484    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 14:32:03.162509 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162492    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 14:32:03.162509 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162506    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 14:32:03.162624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162515    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 14:32:03.162624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162524    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 14:32:03.162624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162530    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 14:32:03.162624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162538    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 14:32:03.162624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162544    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 14:32:03.162624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162561    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 14:32:03.162624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.162567    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 14:32:03.163569 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.163557    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 14:32:03.163569 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.163567    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 14:32:03.166068 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.166049    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pt9xm"
Jul 24 14:32:03.167033 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.167018    2581 server.go:1268] "Started kubelet"
Jul 24 14:32:03.167155 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.167106    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 14:32:03.167365 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.167111    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 14:32:03.167587 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.167546    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 14:32:03.167953 ip-10-0-143-246 systemd[1]: Started Kubernetes Kubelet.
Jul 24 14:32:03.170651 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.170619    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 14:32:03.170879 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.170850    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 14:32:03.171225 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.167699    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-246.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 14:32:03.171323 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.171306    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-246.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:32:03.172895 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.172874    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 14:32:03.173383 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.173364    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pt9xm"
Jul 24 14:32:03.174397 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.174348    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 14:32:03.178616 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.177273    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-246.ec2.internal.18c5404c28b63165  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-246.ec2.internal,UID:ip-10-0-143-246.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-246.ec2.internal,},FirstTimestamp:2026-07-24 14:32:03.166990693 +0000 UTC m=+1.670708648,LastTimestamp:2026-07-24 14:32:03.166990693 +0000 UTC m=+1.670708648,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-246.ec2.internal,}"
Jul 24 14:32:03.185443 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.185424    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 14:32:03.185551 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.185430    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 14:32:03.185609 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.185585    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-246.ec2.internal\" not found"
Jul 24 14:32:03.186190 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.186171    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 14:32:03.186190 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.186169    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 14:32:03.186328 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.186202    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 14:32:03.186328 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.186290    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 14:32:03.186328 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.186303    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 14:32:03.186700 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.186684    2581 factory.go:55] Registering systemd factory
Jul 24 14:32:03.186771 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.186742    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 24 14:32:03.186994 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.186981    2581 factory.go:153] Registering CRI-O factory
Jul 24 14:32:03.187061 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.186996    2581 factory.go:223] Registration of the crio container factory successfully
Jul 24 14:32:03.187061 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.187050    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 14:32:03.187181 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.187075    2581 factory.go:103] Registering Raw factory
Jul 24 14:32:03.187181 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.187092    2581 manager.go:1196] Started watching for new ooms in manager
Jul 24 14:32:03.187677 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.187655    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:32:03.187844 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.187823    2581 manager.go:349] Starting recovery of all containers
Jul 24 14:32:03.190994 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.190970    2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-246.ec2.internal\" not found" node="ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.197199 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.197162    2581 manager.go:354] Recovery completed
Jul 24 14:32:03.201665 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.201650    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:32:03.203613 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.203592    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:32:03.203684 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.203627    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:32:03.203684 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.203641    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:32:03.204239 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.204226    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 14:32:03.204239 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.204236    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 14:32:03.204396 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.204242    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:32:03.204396 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.204253    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 14:32:03.204967 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.204951    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:32:03.205014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.204985    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:32:03.205014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.205001    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:32:03.206083 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.206072    2581 policy_none.go:49] "None policy: Start"
Jul 24 14:32:03.206140 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.206088    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 14:32:03.206140 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.206100    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 14:32:03.207283 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.207272    2581 policy_none.go:47] "Start"
Jul 24 14:32:03.250248 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.250003    2581 manager.go:337] "Starting Device Plugin manager"
Jul 24 14:32:03.250382 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.250274    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 14:32:03.250382 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.250290    2581 server.go:87] "Starting device plugin registration server"
Jul 24 14:32:03.250610 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.250597    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 14:32:03.250642 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.250612    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 14:32:03.250768 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.250740    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 14:32:03.250887 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.250857    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 14:32:03.250887 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.250868    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 14:32:03.251939 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.251916    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 14:32:03.252016 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.251967    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-246.ec2.internal\" not found"
Jul 24 14:32:03.326496 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.326404    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 14:32:03.327777 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.327761    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 14:32:03.327876 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.327796    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 14:32:03.327876 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.327820    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 14:32:03.327876 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.327857    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 14:32:03.330842 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.330822    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:32:03.350902 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.350872    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:32:03.352638 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.351985    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:32:03.352638 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.352022    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:32:03.352638 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.352038    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:32:03.352638 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.352069    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.360906 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.360887    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.361018 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.360915    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-246.ec2.internal\": node \"ip-10-0-143-246.ec2.internal\" not found"
Jul 24 14:32:03.382749 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.382723    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-246.ec2.internal\" not found"
Jul 24 14:32:03.429952 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.429912    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"]
Jul 24 14:32:03.430046 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.430002    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:32:03.431042 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.431023    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:32:03.431112 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.431060    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:32:03.431112 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.431073    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:32:03.433034 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.433020    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:32:03.433203 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.433166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.433203 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.433201    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:32:03.433867 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.433849    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:32:03.433935 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.433880    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:32:03.433935 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.433892    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:32:03.433935 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.433855    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:32:03.434053 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.433959    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:32:03.434053 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.433971    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:32:03.435316 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.435298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.435414 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.435334    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:32:03.436373 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.436349    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:32:03.436463 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.436380    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:32:03.436463 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.436394    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:32:03.457836 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.457809    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-246.ec2.internal\" not found" node="ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.462229 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.462211    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-246.ec2.internal\" not found" node="ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.483731 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.483704    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-246.ec2.internal\" not found"
Jul 24 14:32:03.585824 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.585737    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-246.ec2.internal\" not found"
Jul 24 14:32:03.588200 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.588184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/905224f3fd7ab7503a387859310ffa2d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"905224f3fd7ab7503a387859310ffa2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.588266 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.588209    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/905224f3fd7ab7503a387859310ffa2d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"905224f3fd7ab7503a387859310ffa2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.588266 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.588230    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/346748fe11d636dec2275399a789dab0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-246.ec2.internal\" (UID: \"346748fe11d636dec2275399a789dab0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.686978 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.686934    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-246.ec2.internal\" not found"
Jul 24 14:32:03.689330 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.689310    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/346748fe11d636dec2275399a789dab0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-246.ec2.internal\" (UID: \"346748fe11d636dec2275399a789dab0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.689382 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.689343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/905224f3fd7ab7503a387859310ffa2d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"905224f3fd7ab7503a387859310ffa2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.689382 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.689361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/905224f3fd7ab7503a387859310ffa2d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"905224f3fd7ab7503a387859310ffa2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.689382 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.689368    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/346748fe11d636dec2275399a789dab0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-246.ec2.internal\" (UID: \"346748fe11d636dec2275399a789dab0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.689480 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.689417    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/905224f3fd7ab7503a387859310ffa2d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"905224f3fd7ab7503a387859310ffa2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.689480 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.689417    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/905224f3fd7ab7503a387859310ffa2d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"905224f3fd7ab7503a387859310ffa2d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.760988 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.760938    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.766315 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.766295    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:03.787981 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.787941    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-246.ec2.internal\" not found"
Jul 24 14:32:03.799529 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.799494    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 14:32:03.799708 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.799684    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 14:32:03.799708 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:03.799695    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 14:32:03.891484 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.891376    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-246.ec2.internal\" not found"
Jul 24 14:32:03.991613 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:03.991577    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-246.ec2.internal\" not found"
Jul 24 14:32:04.047691 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.047658    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:32:04.086173 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.086144    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:04.098830 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.098804    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 14:32:04.100715 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.100697    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Jul 24 14:32:04.109031 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.109003    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 14:32:04.156157 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.156067    2581 apiserver.go:52] "Watching apiserver"
Jul 24 14:32:04.163660 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.163632    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 14:32:04.164647 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.164625    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-vqpl5","openshift-network-operator/iptables-alerter-2f654","kube-system/global-pull-secret-syncer-snn2w","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf","openshift-multus/multus-additional-cni-plugins-wh679","openshift-multus/network-metrics-daemon-wxpsq","openshift-ovn-kubernetes/ovnkube-node-4zgrg","kube-system/konnectivity-agent-g44tl","kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal","openshift-cluster-node-tuning-operator/tuned-6vwwl","openshift-image-registry/node-ca-fmzbh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal","openshift-multus/multus-nzj76"]
Jul 24 14:32:04.166804 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.166784    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:04.166878 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.166858    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:04.168880 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.168864    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2f654"
Jul 24 14:32:04.170401 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.170381    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:04.170493 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.170442    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:04.171611 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.171591    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:32:04.171703 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.171650    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 14:32:04.171809 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.171796    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wpl8t\""
Jul 24 14:32:04.171876 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.171841    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.172118 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.171828    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 14:32:04.173851 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.173830    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.174639 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.174617    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 14:32:04.174723 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.174658    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 14:32:04.175624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.175597    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 14:32:04.175818 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.175796    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lmq8g\""
Jul 24 14:32:04.176298 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.176280    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:04.176395 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.176345    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:04.176458 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.176419    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 14:32:04.176936 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.176918    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jv28m\""
Jul 24 14:32:04.177026 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.176965    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 14:32:04.177280 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.177261    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 14:32:04.177455 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.177438    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 14:32:04.177732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.177716    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 14:32:04.179149 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.179119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.179709 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.179643    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 14:27:03 +0000 UTC" deadline="2028-04-07 00:15:13.159356482 +0000 UTC"
Jul 24 14:32:04.179795 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.179711    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14937h43m8.979651699s"
Jul 24 14:32:04.180769 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.180752    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:04.182192 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.182159    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 14:32:04.182288 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.182259    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.182350 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.182306    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 14:32:04.182438 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.182420    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 14:32:04.183000 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.182980    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 14:32:04.183083 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.183026    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 14:32:04.183296 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.183281    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 14:32:04.183528 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.183512    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cd6mt\""
Jul 24 14:32:04.183604 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.183514    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 14:32:04.184263 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.184150    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 14:32:04.184482 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.184465    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fmzbh"
Jul 24 14:32:04.184876 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.184858    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9gs42\""
Jul 24 14:32:04.184958 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.184868    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:32:04.185875 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.185863    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 14:32:04.186491 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.186473    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.186878 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.186862    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 14:32:04.187371 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.187342    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 14:32:04.187462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.187409    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5jqhd\""
Jul 24 14:32:04.187462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.187416    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4k888\""
Jul 24 14:32:04.187462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.187409    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 14:32:04.187690 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.187673    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 14:32:04.189521 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.189496    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 14:32:04.189784 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.189770    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hchwr\""
Jul 24 14:32:04.192284 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192267    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-cni-binary-copy\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.192355 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.192355 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192309    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-system-cni-dir\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.192355 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-cnibin\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.192485 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-var-lib-kubelet\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.192485 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192381    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f983040-8d3c-4f7c-8c82-12586a4033ee-kubelet-config\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:04.192485 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192407    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f983040-8d3c-4f7c-8c82-12586a4033ee-dbus\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:04.192485 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192422    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-run-systemd\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.192485 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192463    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-modprobe-d\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.192644 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192513    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-run-k8s-cni-cncf-io\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.192644 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-cnibin\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.192644 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192559    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flk2r\" (UniqueName: \"kubernetes.io/projected/0e4ffaf2-3ff8-4577-a883-956efc5f007a-kube-api-access-flk2r\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:04.192644 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192574    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa99a100-21cb-48e5-b249-2d3a951bb436-env-overrides\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.192644 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-sysctl-d\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.192644 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192635    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ade776d8-e297-4e5c-b57e-8e6cd942c159-cni-binary-copy\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.192868 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192650    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-etc-kubernetes\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.192868 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:04.192868 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192705    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-kubelet\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.192868 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-cni-bin\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.192868 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-sysctl-conf\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.192868 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-systemd\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.192868 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-host\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.192868 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192808    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d372f67-c893-46a9-8ce9-64507e3ae5ea-serviceca\") pod \"node-ca-fmzbh\" (UID: \"0d372f67-c893-46a9-8ce9-64507e3ae5ea\") " pod="openshift-image-registry/node-ca-fmzbh"
Jul 24 14:32:04.192868 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-var-lib-cni-bin\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192893    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-conf-dir\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6c4n\" (UniqueName: \"kubernetes.io/projected/c3fc1140-588d-41ec-a1d4-50822e44b72e-kube-api-access-t6c4n\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192959    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-slash\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.192998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f98367c-0a6f-491a-a185-8f34de518278-agent-certs\") pod \"konnectivity-agent-g44tl\" (UID: \"7f98367c-0a6f-491a-a185-8f34de518278\") " pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193049    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d372f67-c893-46a9-8ce9-64507e3ae5ea-host\") pod \"node-ca-fmzbh\" (UID: \"0d372f67-c893-46a9-8ce9-64507e3ae5ea\") " pod="openshift-image-registry/node-ca-fmzbh"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193078    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-cni-netd\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193104    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6tpg\" (UniqueName: \"kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg\") pod \"network-check-target-vqpl5\" (UID: \"5ed878d7-e106-4e24-a24b-46253570f910\") " pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193148    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0f4eb038-f797-48f4-8213-b88172a8849e-etc-tuned\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-os-release\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193194    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193216    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-var-lib-kubelet\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-socket-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-registration-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.193301 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193304    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-systemd-units\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193361    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j5fb\" (UniqueName: \"kubernetes.io/projected/aa99a100-21cb-48e5-b249-2d3a951bb436-kube-api-access-2j5fb\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193390    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-system-cni-dir\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-sysconfig\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193441    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-lib-modules\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193473    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-sys-fs\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193524    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-run-ovn\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193557    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa99a100-21cb-48e5-b249-2d3a951bb436-ovn-node-metrics-cert\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa99a100-21cb-48e5-b249-2d3a951bb436-ovnkube-script-lib\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-kubernetes\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193613    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-etc-selinux\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-daemon-config\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193679    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c5b7db0b-7ea5-477f-b451-9cc880feec48-iptables-alerter-script\") pod \"iptables-alerter-2f654\" (UID: \"c5b7db0b-7ea5-477f-b451-9cc880feec48\") " pod="openshift-network-operator/iptables-alerter-2f654"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-var-lib-openvswitch\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.193963 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193750    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-etc-openvswitch\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f4eb038-f797-48f4-8213-b88172a8849e-tmp\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqjsn\" (UniqueName: \"kubernetes.io/projected/0f4eb038-f797-48f4-8213-b88172a8849e-kube-api-access-xqjsn\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vswb8\" (UniqueName: \"kubernetes.io/projected/ade776d8-e297-4e5c-b57e-8e6cd942c159-kube-api-access-vswb8\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193942    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193964    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-device-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.193981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-log-socket\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa99a100-21cb-48e5-b249-2d3a951bb436-ovnkube-config\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194022    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f98367c-0a6f-491a-a185-8f34de518278-konnectivity-ca\") pod \"konnectivity-agent-g44tl\" (UID: \"7f98367c-0a6f-491a-a185-8f34de518278\") " pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194038    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-cni-dir\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194071    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-run-multus-certs\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-run-openvswitch\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194156    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-node-log\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-sys\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194216    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-socket-dir-parent\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194275    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-run-netns\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-run-netns\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.194732 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-run-ovn-kubernetes\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.195264 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194379    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-run\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.195264 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-var-lib-cni-multus\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.195264 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194428    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.195264 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194488    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-os-release\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.195264 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194531    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7gk7\" (UniqueName: \"kubernetes.io/projected/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-kube-api-access-x7gk7\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.195264 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194581    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvwn6\" (UniqueName: \"kubernetes.io/projected/0d372f67-c893-46a9-8ce9-64507e3ae5ea-kube-api-access-qvwn6\") pod \"node-ca-fmzbh\" (UID: \"0d372f67-c893-46a9-8ce9-64507e3ae5ea\") " pod="openshift-image-registry/node-ca-fmzbh"
Jul 24 14:32:04.195264 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194608    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-hostroot\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.195264 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5b7db0b-7ea5-477f-b451-9cc880feec48-host-slash\") pod \"iptables-alerter-2f654\" (UID: \"c5b7db0b-7ea5-477f-b451-9cc880feec48\") " pod="openshift-network-operator/iptables-alerter-2f654"
Jul 24 14:32:04.195264 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.194657    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mg2r\" (UniqueName: \"kubernetes.io/projected/c5b7db0b-7ea5-477f-b451-9cc880feec48-kube-api-access-9mg2r\") pod \"iptables-alerter-2f654\" (UID: \"c5b7db0b-7ea5-477f-b451-9cc880feec48\") " pod="openshift-network-operator/iptables-alerter-2f654"
Jul 24 14:32:04.201033 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.201012    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 14:32:04.224322 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.224299    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7zb4m"
Jul 24 14:32:04.233820 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.233798    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7zb4m"
Jul 24 14:32:04.237910 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.237887    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:32:04.287579 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.287556    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 14:32:04.294868 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.294842    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-os-release\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.294993 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.294886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.294993 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.294918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-var-lib-kubelet\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.295153 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.294988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-socket-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.295153 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295003    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-os-release\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.295153 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295028    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-registration-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.295153 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295008    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-var-lib-kubelet\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.295153 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295087    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-registration-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.295153 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295141    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-systemd-units\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.295421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.295421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295182    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-socket-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.295421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295253    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-systemd-units\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.295741 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295716    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.295888 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.295941 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j5fb\" (UniqueName: \"kubernetes.io/projected/aa99a100-21cb-48e5-b249-2d3a951bb436-kube-api-access-2j5fb\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.295996 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-system-cni-dir\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.296047 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.295998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-sysconfig\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.296047 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-lib-modules\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.296229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296059    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-sys-fs\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.296229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296082    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-system-cni-dir\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.296229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296084    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.296229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296159    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-sys-fs\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.296229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296166    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-run-ovn\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.296229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296204    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-sysconfig\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.296229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-lib-modules\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296234    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-run-ovn\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296252    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa99a100-21cb-48e5-b249-2d3a951bb436-ovn-node-metrics-cert\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa99a100-21cb-48e5-b249-2d3a951bb436-ovnkube-script-lib\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296309    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-kubernetes\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296354    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-etc-selinux\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296382    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-daemon-config\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296402    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c5b7db0b-7ea5-477f-b451-9cc880feec48-iptables-alerter-script\") pod \"iptables-alerter-2f654\" (UID: \"c5b7db0b-7ea5-477f-b451-9cc880feec48\") " pod="openshift-network-operator/iptables-alerter-2f654"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296422    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-var-lib-openvswitch\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-etc-openvswitch\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296450    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-etc-selinux\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296356    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-kubernetes\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f4eb038-f797-48f4-8213-b88172a8849e-tmp\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqjsn\" (UniqueName: \"kubernetes.io/projected/0f4eb038-f797-48f4-8213-b88172a8849e-kube-api-access-xqjsn\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vswb8\" (UniqueName: \"kubernetes.io/projected/ade776d8-e297-4e5c-b57e-8e6cd942c159-kube-api-access-vswb8\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.296583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296594    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-device-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-log-socket\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa99a100-21cb-48e5-b249-2d3a951bb436-ovnkube-config\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f98367c-0a6f-491a-a185-8f34de518278-konnectivity-ca\") pod \"konnectivity-agent-g44tl\" (UID: \"7f98367c-0a6f-491a-a185-8f34de518278\") " pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-cni-dir\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-run-multus-certs\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296776    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-run-openvswitch\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296842    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-node-log\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-sys\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-socket-dir-parent\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-run-netns\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa99a100-21cb-48e5-b249-2d3a951bb436-ovnkube-script-lib\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-run-netns\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.296993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-run-ovn-kubernetes\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297026    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-etc-openvswitch\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-run\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.297421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-var-lib-cni-multus\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297084    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-device-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297099    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-os-release\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297170    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-daemon-config\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297194    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-cni-dir\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gk7\" (UniqueName: \"kubernetes.io/projected/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-kube-api-access-x7gk7\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvwn6\" (UniqueName: \"kubernetes.io/projected/0d372f67-c893-46a9-8ce9-64507e3ae5ea-kube-api-access-qvwn6\") pod \"node-ca-fmzbh\" (UID: \"0d372f67-c893-46a9-8ce9-64507e3ae5ea\") " pod="openshift-image-registry/node-ca-fmzbh"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297276    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-hostroot\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-log-socket\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297407    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c5b7db0b-7ea5-477f-b451-9cc880feec48-iptables-alerter-script\") pod \"iptables-alerter-2f654\" (UID: \"c5b7db0b-7ea5-477f-b451-9cc880feec48\") " pod="openshift-network-operator/iptables-alerter-2f654"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.297455    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297487    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-run-ovn-kubernetes\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-run-multus-certs\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-run-openvswitch\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-node-log\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.297672    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret podName:8f983040-8d3c-4f7c-8c82-12586a4033ee nodeName:}" failed. No retries permitted until 2026-07-24 14:32:04.797613286 +0000 UTC m=+3.301331238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret") pod "global-pull-secret-syncer-snn2w" (UID: "8f983040-8d3c-4f7c-8c82-12586a4033ee") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:04.298276 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa99a100-21cb-48e5-b249-2d3a951bb436-ovnkube-config\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297703    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-sys\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297752    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-run-netns\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297762    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c3fc1140-588d-41ec-a1d4-50822e44b72e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f98367c-0a6f-491a-a185-8f34de518278-konnectivity-ca\") pod \"konnectivity-agent-g44tl\" (UID: \"7f98367c-0a6f-491a-a185-8f34de518278\") " pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-run\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-var-lib-openvswitch\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297795    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-run-netns\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297861    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-var-lib-cni-multus\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297882    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-hostroot\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297875    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-socket-dir-parent\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297941    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-os-release\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.297970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5b7db0b-7ea5-477f-b451-9cc880feec48-host-slash\") pod \"iptables-alerter-2f654\" (UID: \"c5b7db0b-7ea5-477f-b451-9cc880feec48\") " pod="openshift-network-operator/iptables-alerter-2f654"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298011    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5b7db0b-7ea5-477f-b451-9cc880feec48-host-slash\") pod \"iptables-alerter-2f654\" (UID: \"c5b7db0b-7ea5-477f-b451-9cc880feec48\") " pod="openshift-network-operator/iptables-alerter-2f654"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mg2r\" (UniqueName: \"kubernetes.io/projected/c5b7db0b-7ea5-477f-b451-9cc880feec48-kube-api-access-9mg2r\") pod \"iptables-alerter-2f654\" (UID: \"c5b7db0b-7ea5-477f-b451-9cc880feec48\") " pod="openshift-network-operator/iptables-alerter-2f654"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298296    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-cni-binary-copy\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-system-cni-dir\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.299292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298400    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-cnibin\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-var-lib-kubelet\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f983040-8d3c-4f7c-8c82-12586a4033ee-kubelet-config\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f983040-8d3c-4f7c-8c82-12586a4033ee-dbus\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-run-systemd\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-modprobe-d\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298565    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-run-k8s-cni-cncf-io\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-cnibin\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298654    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flk2r\" (UniqueName: \"kubernetes.io/projected/0e4ffaf2-3ff8-4577-a883-956efc5f007a-kube-api-access-flk2r\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa99a100-21cb-48e5-b249-2d3a951bb436-env-overrides\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f983040-8d3c-4f7c-8c82-12586a4033ee-dbus\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-sysctl-d\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ade776d8-e297-4e5c-b57e-8e6cd942c159-cni-binary-copy\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298812    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-system-cni-dir\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-etc-kubernetes\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298878    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-cnibin\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-var-lib-kubelet\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298937    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-cni-binary-copy\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.300229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.299423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f983040-8d3c-4f7c-8c82-12586a4033ee-kubelet-config\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.299530    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-run-systemd\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.299668    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-sysctl-d\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.299754    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.299954    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs podName:0e4ffaf2-3ff8-4577-a883-956efc5f007a nodeName:}" failed. No retries permitted until 2026-07-24 14:32:04.799908191 +0000 UTC m=+3.303626136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs") pod "network-metrics-daemon-wxpsq" (UID: "0e4ffaf2-3ff8-4577-a883-956efc5f007a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.300230    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-run-k8s-cni-cncf-io\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.300361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-modprobe-d\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.300385    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ade776d8-e297-4e5c-b57e-8e6cd942c159-cni-binary-copy\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.300425    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-cnibin\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.300438    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa99a100-21cb-48e5-b249-2d3a951bb436-env-overrides\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.300457    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-etc-kubernetes\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.300855    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f4eb038-f797-48f4-8213-b88172a8849e-tmp\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298958    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:04.301120 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.301067    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa99a100-21cb-48e5-b249-2d3a951bb436-ovn-node-metrics-cert\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.301803 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.301069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-kubelet\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.301803 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.298946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.301803 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.301220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-cni-bin\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.301803 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.301130    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-kubelet\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-sysctl-conf\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-systemd\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-host\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d372f67-c893-46a9-8ce9-64507e3ae5ea-serviceca\") pod \"node-ca-fmzbh\" (UID: \"0d372f67-c893-46a9-8ce9-64507e3ae5ea\") " pod="openshift-image-registry/node-ca-fmzbh"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-var-lib-cni-bin\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-conf-dir\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302448    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-host-var-lib-cni-bin\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ade776d8-e297-4e5c-b57e-8e6cd942c159-multus-conf-dir\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302520    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-cni-bin\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6c4n\" (UniqueName: \"kubernetes.io/projected/c3fc1140-588d-41ec-a1d4-50822e44b72e-kube-api-access-t6c4n\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302593    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-slash\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302640    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-host\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302652    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-sysctl-conf\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302730    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f98367c-0a6f-491a-a185-8f34de518278-agent-certs\") pod \"konnectivity-agent-g44tl\" (UID: \"7f98367c-0a6f-491a-a185-8f34de518278\") " pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302763    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d372f67-c893-46a9-8ce9-64507e3ae5ea-host\") pod \"node-ca-fmzbh\" (UID: \"0d372f67-c893-46a9-8ce9-64507e3ae5ea\") " pod="openshift-image-registry/node-ca-fmzbh"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-cni-netd\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6tpg\" (UniqueName: \"kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg\") pod \"network-check-target-vqpl5\" (UID: \"5ed878d7-e106-4e24-a24b-46253570f910\") " pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:04.303342 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.302996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0f4eb038-f797-48f4-8213-b88172a8849e-etc-tuned\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.304471 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.303457    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-slash\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.304471 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.303687    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d372f67-c893-46a9-8ce9-64507e3ae5ea-serviceca\") pod \"node-ca-fmzbh\" (UID: \"0d372f67-c893-46a9-8ce9-64507e3ae5ea\") " pod="openshift-image-registry/node-ca-fmzbh"
Jul 24 14:32:04.304471 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.304301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d372f67-c893-46a9-8ce9-64507e3ae5ea-host\") pod \"node-ca-fmzbh\" (UID: \"0d372f67-c893-46a9-8ce9-64507e3ae5ea\") " pod="openshift-image-registry/node-ca-fmzbh"
Jul 24 14:32:04.305890 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.304652    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa99a100-21cb-48e5-b249-2d3a951bb436-host-cni-netd\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.305890 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.305039    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0f4eb038-f797-48f4-8213-b88172a8849e-etc-systemd\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.307919 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.307191    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0f4eb038-f797-48f4-8213-b88172a8849e-etc-tuned\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.307919 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.307474    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j5fb\" (UniqueName: \"kubernetes.io/projected/aa99a100-21cb-48e5-b249-2d3a951bb436-kube-api-access-2j5fb\") pod \"ovnkube-node-4zgrg\" (UID: \"aa99a100-21cb-48e5-b249-2d3a951bb436\") " pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.308468 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.308401    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f98367c-0a6f-491a-a185-8f34de518278-agent-certs\") pod \"konnectivity-agent-g44tl\" (UID: \"7f98367c-0a6f-491a-a185-8f34de518278\") " pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:04.309024 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.308998    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:32:04.309160 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.309031    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:32:04.309160 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.309051    2581 projected.go:196] Error preparing data for projected volume kube-api-access-b6tpg for pod openshift-network-diagnostics/network-check-target-vqpl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:04.309333 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.309215    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg podName:5ed878d7-e106-4e24-a24b-46253570f910 nodeName:}" failed. No retries permitted until 2026-07-24 14:32:04.809169072 +0000 UTC m=+3.312887029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b6tpg" (UniqueName: "kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg") pod "network-check-target-vqpl5" (UID: "5ed878d7-e106-4e24-a24b-46253570f910") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:04.310746 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.310697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvwn6\" (UniqueName: \"kubernetes.io/projected/0d372f67-c893-46a9-8ce9-64507e3ae5ea-kube-api-access-qvwn6\") pod \"node-ca-fmzbh\" (UID: \"0d372f67-c893-46a9-8ce9-64507e3ae5ea\") " pod="openshift-image-registry/node-ca-fmzbh"
Jul 24 14:32:04.310746 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.310727    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flk2r\" (UniqueName: \"kubernetes.io/projected/0e4ffaf2-3ff8-4577-a883-956efc5f007a-kube-api-access-flk2r\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:04.310908 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.310728    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7gk7\" (UniqueName: \"kubernetes.io/projected/c1aafd01-1d92-4f6e-b4d3-aceee2f33332-kube-api-access-x7gk7\") pod \"multus-additional-cni-plugins-wh679\" (UID: \"c1aafd01-1d92-4f6e-b4d3-aceee2f33332\") " pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.311205 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.311109    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vswb8\" (UniqueName: \"kubernetes.io/projected/ade776d8-e297-4e5c-b57e-8e6cd942c159-kube-api-access-vswb8\") pod \"multus-nzj76\" (UID: \"ade776d8-e297-4e5c-b57e-8e6cd942c159\") " pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.311282 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.311204    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqjsn\" (UniqueName: \"kubernetes.io/projected/0f4eb038-f797-48f4-8213-b88172a8849e-kube-api-access-xqjsn\") pod \"tuned-6vwwl\" (UID: \"0f4eb038-f797-48f4-8213-b88172a8849e\") " pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.311928 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.311906    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mg2r\" (UniqueName: \"kubernetes.io/projected/c5b7db0b-7ea5-477f-b451-9cc880feec48-kube-api-access-9mg2r\") pod \"iptables-alerter-2f654\" (UID: \"c5b7db0b-7ea5-477f-b451-9cc880feec48\") " pod="openshift-network-operator/iptables-alerter-2f654"
Jul 24 14:32:04.312445 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.312424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6c4n\" (UniqueName: \"kubernetes.io/projected/c3fc1140-588d-41ec-a1d4-50822e44b72e-kube-api-access-t6c4n\") pod \"aws-ebs-csi-driver-node-chbwf\" (UID: \"c3fc1140-588d-41ec-a1d4-50822e44b72e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.377847 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:04.377800    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod905224f3fd7ab7503a387859310ffa2d.slice/crio-ebc7cd1bccc55949c4dcf7626bd5c03772776918b9eb73347338831501ec1f3d WatchSource:0}: Error finding container ebc7cd1bccc55949c4dcf7626bd5c03772776918b9eb73347338831501ec1f3d: Status 404 returned error can't find the container with id ebc7cd1bccc55949c4dcf7626bd5c03772776918b9eb73347338831501ec1f3d
Jul 24 14:32:04.378302 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:04.378278    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346748fe11d636dec2275399a789dab0.slice/crio-1b9e4f93e97657d6fed13adcfd53b9baff9b9c88dfb1f3ed6360f0acb4aa8b6a WatchSource:0}: Error finding container 1b9e4f93e97657d6fed13adcfd53b9baff9b9c88dfb1f3ed6360f0acb4aa8b6a: Status 404 returned error can't find the container with id 1b9e4f93e97657d6fed13adcfd53b9baff9b9c88dfb1f3ed6360f0acb4aa8b6a
Jul 24 14:32:04.382646 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.382627    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 14:32:04.486242 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.486146    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2f654"
Jul 24 14:32:04.496465 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:04.496434    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b7db0b_7ea5_477f_b451_9cc880feec48.slice/crio-b0c1a2105265bfeef2a9a9092b30eae8873584d17260bf9bb423b67f7c764c9d WatchSource:0}: Error finding container b0c1a2105265bfeef2a9a9092b30eae8873584d17260bf9bb423b67f7c764c9d: Status 404 returned error can't find the container with id b0c1a2105265bfeef2a9a9092b30eae8873584d17260bf9bb423b67f7c764c9d
Jul 24 14:32:04.503209 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.503187    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf"
Jul 24 14:32:04.511856 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:04.511832    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3fc1140_588d_41ec_a1d4_50822e44b72e.slice/crio-bb98701cb5e73fdaf896ab77c30e8181a99c596350b9844880e6475716e85536 WatchSource:0}: Error finding container bb98701cb5e73fdaf896ab77c30e8181a99c596350b9844880e6475716e85536: Status 404 returned error can't find the container with id bb98701cb5e73fdaf896ab77c30e8181a99c596350b9844880e6475716e85536
Jul 24 14:32:04.520207 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.520189    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wh679"
Jul 24 14:32:04.529003 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:04.528983    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1aafd01_1d92_4f6e_b4d3_aceee2f33332.slice/crio-5d10ba5e8b8769d458d49aeb06b877309428a0faa8283d15ae8262a8ca7217b6 WatchSource:0}: Error finding container 5d10ba5e8b8769d458d49aeb06b877309428a0faa8283d15ae8262a8ca7217b6: Status 404 returned error can't find the container with id 5d10ba5e8b8769d458d49aeb06b877309428a0faa8283d15ae8262a8ca7217b6
Jul 24 14:32:04.538268 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.538249    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:04.546130 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:04.546108    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa99a100_21cb_48e5_b249_2d3a951bb436.slice/crio-8176458045c30d1d4590865f5e13f634b4001b9e8142264c9bd48352385864b4 WatchSource:0}: Error finding container 8176458045c30d1d4590865f5e13f634b4001b9e8142264c9bd48352385864b4: Status 404 returned error can't find the container with id 8176458045c30d1d4590865f5e13f634b4001b9e8142264c9bd48352385864b4
Jul 24 14:32:04.553296 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.553273    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:04.561366 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.561341    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6vwwl"
Jul 24 14:32:04.561661 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:04.561642    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f98367c_0a6f_491a_a185_8f34de518278.slice/crio-833ed28c363f4ebf91fe046b319f6c5bd424b6c20f8adb77e8ed38203bca0a68 WatchSource:0}: Error finding container 833ed28c363f4ebf91fe046b319f6c5bd424b6c20f8adb77e8ed38203bca0a68: Status 404 returned error can't find the container with id 833ed28c363f4ebf91fe046b319f6c5bd424b6c20f8adb77e8ed38203bca0a68
Jul 24 14:32:04.569008 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:04.568989    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f4eb038_f797_48f4_8213_b88172a8849e.slice/crio-b48130a656dd725678d0383b110876b27a261171704f5c321bcdecdf26e99379 WatchSource:0}: Error finding container b48130a656dd725678d0383b110876b27a261171704f5c321bcdecdf26e99379: Status 404 returned error can't find the container with id b48130a656dd725678d0383b110876b27a261171704f5c321bcdecdf26e99379
Jul 24 14:32:04.591495 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.591468    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fmzbh"
Jul 24 14:32:04.597228 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.597090    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nzj76"
Jul 24 14:32:04.599531 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:04.599511    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d372f67_c893_46a9_8ce9_64507e3ae5ea.slice/crio-6d88db92ba55c4818b893be50944adf00b48496c233d50d9bd6721d6da754dad WatchSource:0}: Error finding container 6d88db92ba55c4818b893be50944adf00b48496c233d50d9bd6721d6da754dad: Status 404 returned error can't find the container with id 6d88db92ba55c4818b893be50944adf00b48496c233d50d9bd6721d6da754dad
Jul 24 14:32:04.605004 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:04.604984    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade776d8_e297_4e5c_b57e_8e6cd942c159.slice/crio-893982e8e6c74d3ddca19ab26e7c6b34ac1c18d48ff0684028e2d6a21bf84f45 WatchSource:0}: Error finding container 893982e8e6c74d3ddca19ab26e7c6b34ac1c18d48ff0684028e2d6a21bf84f45: Status 404 returned error can't find the container with id 893982e8e6c74d3ddca19ab26e7c6b34ac1c18d48ff0684028e2d6a21bf84f45
Jul 24 14:32:04.806549 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.806459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:04.806549 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.806508    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:04.806776 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.806631    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:04.806776 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.806632    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:04.806776 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.806692    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs podName:0e4ffaf2-3ff8-4577-a883-956efc5f007a nodeName:}" failed. No retries permitted until 2026-07-24 14:32:05.806673512 +0000 UTC m=+4.310391475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs") pod "network-metrics-daemon-wxpsq" (UID: "0e4ffaf2-3ff8-4577-a883-956efc5f007a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:04.806776 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.806707    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret podName:8f983040-8d3c-4f7c-8c82-12586a4033ee nodeName:}" failed. No retries permitted until 2026-07-24 14:32:05.806700093 +0000 UTC m=+4.310418030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret") pod "global-pull-secret-syncer-snn2w" (UID: "8f983040-8d3c-4f7c-8c82-12586a4033ee") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:04.907585 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.907551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6tpg\" (UniqueName: \"kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg\") pod \"network-check-target-vqpl5\" (UID: \"5ed878d7-e106-4e24-a24b-46253570f910\") " pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:04.907790 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.907755    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:32:04.907790 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.907781    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:32:04.907931 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.907795    2581 projected.go:196] Error preparing data for projected volume kube-api-access-b6tpg for pod openshift-network-diagnostics/network-check-target-vqpl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:04.907931 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:04.907867    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg podName:5ed878d7-e106-4e24-a24b-46253570f910 nodeName:}" failed. No retries permitted until 2026-07-24 14:32:05.907842999 +0000 UTC m=+4.411560939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6tpg" (UniqueName: "kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg") pod "network-check-target-vqpl5" (UID: "5ed878d7-e106-4e24-a24b-46253570f910") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:04.937281 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:04.937071    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:32:05.236558 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.236457    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 14:27:04 +0000 UTC" deadline="2028-03-14 01:44:09.999359646 +0000 UTC"
Jul 24 14:32:05.236558 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.236505    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14363h12m4.762860487s"
Jul 24 14:32:05.254724 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.254690    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:32:05.339015 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.338953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal" event={"ID":"346748fe11d636dec2275399a789dab0","Type":"ContainerStarted","Data":"1b9e4f93e97657d6fed13adcfd53b9baff9b9c88dfb1f3ed6360f0acb4aa8b6a"}
Jul 24 14:32:05.344418 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.344364    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fmzbh" event={"ID":"0d372f67-c893-46a9-8ce9-64507e3ae5ea","Type":"ContainerStarted","Data":"6d88db92ba55c4818b893be50944adf00b48496c233d50d9bd6721d6da754dad"}
Jul 24 14:32:05.352034 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.351992    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6vwwl" event={"ID":"0f4eb038-f797-48f4-8213-b88172a8849e","Type":"ContainerStarted","Data":"b48130a656dd725678d0383b110876b27a261171704f5c321bcdecdf26e99379"}
Jul 24 14:32:05.361329 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.361280    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g44tl" event={"ID":"7f98367c-0a6f-491a-a185-8f34de518278","Type":"ContainerStarted","Data":"833ed28c363f4ebf91fe046b319f6c5bd424b6c20f8adb77e8ed38203bca0a68"}
Jul 24 14:32:05.373334 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.373231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg" event={"ID":"aa99a100-21cb-48e5-b249-2d3a951bb436","Type":"ContainerStarted","Data":"8176458045c30d1d4590865f5e13f634b4001b9e8142264c9bd48352385864b4"}
Jul 24 14:32:05.376243 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.376163    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wh679" event={"ID":"c1aafd01-1d92-4f6e-b4d3-aceee2f33332","Type":"ContainerStarted","Data":"5d10ba5e8b8769d458d49aeb06b877309428a0faa8283d15ae8262a8ca7217b6"}
Jul 24 14:32:05.387112 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.387072    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf" event={"ID":"c3fc1140-588d-41ec-a1d4-50822e44b72e","Type":"ContainerStarted","Data":"bb98701cb5e73fdaf896ab77c30e8181a99c596350b9844880e6475716e85536"}
Jul 24 14:32:05.389710 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.389655    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2f654" event={"ID":"c5b7db0b-7ea5-477f-b451-9cc880feec48","Type":"ContainerStarted","Data":"b0c1a2105265bfeef2a9a9092b30eae8873584d17260bf9bb423b67f7c764c9d"}
Jul 24 14:32:05.397244 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.397178    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nzj76" event={"ID":"ade776d8-e297-4e5c-b57e-8e6cd942c159","Type":"ContainerStarted","Data":"893982e8e6c74d3ddca19ab26e7c6b34ac1c18d48ff0684028e2d6a21bf84f45"}
Jul 24 14:32:05.403582 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.403523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" event={"ID":"905224f3fd7ab7503a387859310ffa2d","Type":"ContainerStarted","Data":"ebc7cd1bccc55949c4dcf7626bd5c03772776918b9eb73347338831501ec1f3d"}
Jul 24 14:32:05.818892 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.818036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:05.818892 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.818140    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:05.818892 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:05.818282    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:05.818892 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:05.818350    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret podName:8f983040-8d3c-4f7c-8c82-12586a4033ee nodeName:}" failed. No retries permitted until 2026-07-24 14:32:07.818329898 +0000 UTC m=+6.322047838 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret") pod "global-pull-secret-syncer-snn2w" (UID: "8f983040-8d3c-4f7c-8c82-12586a4033ee") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:05.818892 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:05.818795    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:05.818892 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:05.818851    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs podName:0e4ffaf2-3ff8-4577-a883-956efc5f007a nodeName:}" failed. No retries permitted until 2026-07-24 14:32:07.818834392 +0000 UTC m=+6.322552330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs") pod "network-metrics-daemon-wxpsq" (UID: "0e4ffaf2-3ff8-4577-a883-956efc5f007a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:05.919069 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:05.919010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6tpg\" (UniqueName: \"kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg\") pod \"network-check-target-vqpl5\" (UID: \"5ed878d7-e106-4e24-a24b-46253570f910\") " pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:05.919287 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:05.919245    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:32:05.919287 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:05.919269    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:32:05.919287 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:05.919282    2581 projected.go:196] Error preparing data for projected volume kube-api-access-b6tpg for pod openshift-network-diagnostics/network-check-target-vqpl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:05.919456 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:05.919357    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg podName:5ed878d7-e106-4e24-a24b-46253570f910 nodeName:}" failed. No retries permitted until 2026-07-24 14:32:07.919336661 +0000 UTC m=+6.423054620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6tpg" (UniqueName: "kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg") pod "network-check-target-vqpl5" (UID: "5ed878d7-e106-4e24-a24b-46253570f910") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:06.237337 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:06.237269    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 14:27:04 +0000 UTC" deadline="2028-04-28 02:17:24.539302946 +0000 UTC"
Jul 24 14:32:06.237337 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:06.237334    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15443h45m18.301972809s"
Jul 24 14:32:06.328500 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:06.328461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:06.328690 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:06.328609    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:06.329114 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:06.329071    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:06.329244 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:06.329219    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:06.329320 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:06.329305    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:06.329428 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:06.329387    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:07.837437 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:07.837395    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:07.837925 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:07.837576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:07.837925 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:07.837596    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:07.837925 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:07.837689    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs podName:0e4ffaf2-3ff8-4577-a883-956efc5f007a nodeName:}" failed. No retries permitted until 2026-07-24 14:32:11.837663041 +0000 UTC m=+10.341380988 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs") pod "network-metrics-daemon-wxpsq" (UID: "0e4ffaf2-3ff8-4577-a883-956efc5f007a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:07.837925 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:07.837716    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:07.837925 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:07.837779    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret podName:8f983040-8d3c-4f7c-8c82-12586a4033ee nodeName:}" failed. No retries permitted until 2026-07-24 14:32:11.83776268 +0000 UTC m=+10.341480621 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret") pod "global-pull-secret-syncer-snn2w" (UID: "8f983040-8d3c-4f7c-8c82-12586a4033ee") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:07.938702 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:07.938651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6tpg\" (UniqueName: \"kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg\") pod \"network-check-target-vqpl5\" (UID: \"5ed878d7-e106-4e24-a24b-46253570f910\") " pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:07.938921 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:07.938901    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:32:07.939001 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:07.938931    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:32:07.939001 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:07.938945    2581 projected.go:196] Error preparing data for projected volume kube-api-access-b6tpg for pod openshift-network-diagnostics/network-check-target-vqpl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:07.939118 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:07.939005    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg podName:5ed878d7-e106-4e24-a24b-46253570f910 nodeName:}" failed. No retries permitted until 2026-07-24 14:32:11.938985273 +0000 UTC m=+10.442703227 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6tpg" (UniqueName: "kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg") pod "network-check-target-vqpl5" (UID: "5ed878d7-e106-4e24-a24b-46253570f910") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:08.329605 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:08.328715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:08.329605 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:08.328873    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:08.329605 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:08.329286    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:08.329605 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:08.329381    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:08.329605 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:08.329429    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:08.329605 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:08.329508    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:10.328857 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:10.328814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:10.329391 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:10.328860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:10.329391 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:10.328819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:10.329391 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:10.328956    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:10.329391 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:10.329056    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:10.329391 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:10.329172    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:11.874802 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:11.874756    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:11.875286 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:11.874850    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:11.875286 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:11.874939    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:11.875286 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:11.874990    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:11.875286 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:11.875026    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret podName:8f983040-8d3c-4f7c-8c82-12586a4033ee nodeName:}" failed. No retries permitted until 2026-07-24 14:32:19.875001426 +0000 UTC m=+18.378719381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret") pod "global-pull-secret-syncer-snn2w" (UID: "8f983040-8d3c-4f7c-8c82-12586a4033ee") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:11.875286 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:11.875049    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs podName:0e4ffaf2-3ff8-4577-a883-956efc5f007a nodeName:}" failed. No retries permitted until 2026-07-24 14:32:19.875036844 +0000 UTC m=+18.378754789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs") pod "network-metrics-daemon-wxpsq" (UID: "0e4ffaf2-3ff8-4577-a883-956efc5f007a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:11.975478 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:11.975437    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6tpg\" (UniqueName: \"kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg\") pod \"network-check-target-vqpl5\" (UID: \"5ed878d7-e106-4e24-a24b-46253570f910\") " pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:11.975667 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:11.975641    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:32:11.975742 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:11.975670    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:32:11.975742 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:11.975685    2581 projected.go:196] Error preparing data for projected volume kube-api-access-b6tpg for pod openshift-network-diagnostics/network-check-target-vqpl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:11.975854 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:11.975750    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg podName:5ed878d7-e106-4e24-a24b-46253570f910 nodeName:}" failed. No retries permitted until 2026-07-24 14:32:19.975734239 +0000 UTC m=+18.479452178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6tpg" (UniqueName: "kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg") pod "network-check-target-vqpl5" (UID: "5ed878d7-e106-4e24-a24b-46253570f910") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:12.328432 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:12.328341    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:12.328432 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:12.328371    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:12.328432 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:12.328371    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:12.328719 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:12.328502    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:12.328719 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:12.328633    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:12.328836 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:12.328731    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:14.328715 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:14.328627    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:14.329212 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:14.328781    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:14.329212 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:14.328794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:14.329212 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:14.328882    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:14.329212 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:14.328914    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:14.329212 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:14.328975    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:16.328183 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:16.328147    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:16.328626 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:16.328153    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:16.328626 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:16.328261    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:16.328626 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:16.328357    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:16.328626 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:16.328153    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:16.328626 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:16.328477    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:18.328742 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:18.328704    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:18.329156 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:18.328747    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:18.329156 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:18.328822    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:18.329156 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:18.328843    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:18.329156 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:18.328910    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:18.329156 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:18.328969    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:19.939220 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:19.939165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:19.939677 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:19.939244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:19.939677 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:19.939371    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:19.939677 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:19.939433    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs podName:0e4ffaf2-3ff8-4577-a883-956efc5f007a nodeName:}" failed. No retries permitted until 2026-07-24 14:32:35.939414483 +0000 UTC m=+34.443132421 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs") pod "network-metrics-daemon-wxpsq" (UID: "0e4ffaf2-3ff8-4577-a883-956efc5f007a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:19.939677 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:19.939368    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:19.939677 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:19.939514    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret podName:8f983040-8d3c-4f7c-8c82-12586a4033ee nodeName:}" failed. No retries permitted until 2026-07-24 14:32:35.93949918 +0000 UTC m=+34.443217116 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret") pod "global-pull-secret-syncer-snn2w" (UID: "8f983040-8d3c-4f7c-8c82-12586a4033ee") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:20.039596 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.039561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6tpg\" (UniqueName: \"kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg\") pod \"network-check-target-vqpl5\" (UID: \"5ed878d7-e106-4e24-a24b-46253570f910\") " pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:20.039738 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:20.039700    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:32:20.039738 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:20.039720    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:32:20.039738 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:20.039730    2581 projected.go:196] Error preparing data for projected volume kube-api-access-b6tpg for pod openshift-network-diagnostics/network-check-target-vqpl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:20.039836 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:20.039779    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg podName:5ed878d7-e106-4e24-a24b-46253570f910 nodeName:}" failed. No retries permitted until 2026-07-24 14:32:36.039765775 +0000 UTC m=+34.543483712 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6tpg" (UniqueName: "kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg") pod "network-check-target-vqpl5" (UID: "5ed878d7-e106-4e24-a24b-46253570f910") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:20.195360 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.195282    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-fplzf"]
Jul 24 14:32:20.209377 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.209349    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fplzf"
Jul 24 14:32:20.212004 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.211972    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sh44r\""
Jul 24 14:32:20.212004 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.211995    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 14:32:20.212206 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.212104    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 14:32:20.329108 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.329062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:20.329312 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.329114    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:20.329312 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:20.329213    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:20.329312 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.329252    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:20.329485 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:20.329367    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:20.329485 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:20.329472    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:20.341439 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.341412    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7500633c-648e-4a58-bb81-54bd2d438612-tmp-dir\") pod \"node-resolver-fplzf\" (UID: \"7500633c-648e-4a58-bb81-54bd2d438612\") " pod="openshift-dns/node-resolver-fplzf"
Jul 24 14:32:20.341577 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.341472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7500633c-648e-4a58-bb81-54bd2d438612-hosts-file\") pod \"node-resolver-fplzf\" (UID: \"7500633c-648e-4a58-bb81-54bd2d438612\") " pod="openshift-dns/node-resolver-fplzf"
Jul 24 14:32:20.341628 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.341554    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mklf9\" (UniqueName: \"kubernetes.io/projected/7500633c-648e-4a58-bb81-54bd2d438612-kube-api-access-mklf9\") pod \"node-resolver-fplzf\" (UID: \"7500633c-648e-4a58-bb81-54bd2d438612\") " pod="openshift-dns/node-resolver-fplzf"
Jul 24 14:32:20.442589 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.442559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7500633c-648e-4a58-bb81-54bd2d438612-tmp-dir\") pod \"node-resolver-fplzf\" (UID: \"7500633c-648e-4a58-bb81-54bd2d438612\") " pod="openshift-dns/node-resolver-fplzf"
Jul 24 14:32:20.442834 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.442728    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7500633c-648e-4a58-bb81-54bd2d438612-hosts-file\") pod \"node-resolver-fplzf\" (UID: \"7500633c-648e-4a58-bb81-54bd2d438612\") " pod="openshift-dns/node-resolver-fplzf"
Jul 24 14:32:20.442834 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.442797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mklf9\" (UniqueName: \"kubernetes.io/projected/7500633c-648e-4a58-bb81-54bd2d438612-kube-api-access-mklf9\") pod \"node-resolver-fplzf\" (UID: \"7500633c-648e-4a58-bb81-54bd2d438612\") " pod="openshift-dns/node-resolver-fplzf"
Jul 24 14:32:20.442953 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.442842    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7500633c-648e-4a58-bb81-54bd2d438612-hosts-file\") pod \"node-resolver-fplzf\" (UID: \"7500633c-648e-4a58-bb81-54bd2d438612\") " pod="openshift-dns/node-resolver-fplzf"
Jul 24 14:32:20.443005 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.442949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7500633c-648e-4a58-bb81-54bd2d438612-tmp-dir\") pod \"node-resolver-fplzf\" (UID: \"7500633c-648e-4a58-bb81-54bd2d438612\") " pod="openshift-dns/node-resolver-fplzf"
Jul 24 14:32:20.454345 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.454274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mklf9\" (UniqueName: \"kubernetes.io/projected/7500633c-648e-4a58-bb81-54bd2d438612-kube-api-access-mklf9\") pod \"node-resolver-fplzf\" (UID: \"7500633c-648e-4a58-bb81-54bd2d438612\") " pod="openshift-dns/node-resolver-fplzf"
Jul 24 14:32:20.519456 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:20.519409    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fplzf"
Jul 24 14:32:22.328136 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:22.328082    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:22.328590 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:22.328112    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:22.328590 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:22.328216    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:22.328590 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:22.328325    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:22.328590 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:22.328113    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:22.328590 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:22.328427    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:24.328538 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:24.328386    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:24.328843 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:24.328441    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:24.328843 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:24.328607    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:24.328843 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:24.328441    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:24.328843 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:24.328734    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:24.328843 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:24.328817    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:24.452473 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:24.451989    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6vwwl" event={"ID":"0f4eb038-f797-48f4-8213-b88172a8849e","Type":"ContainerStarted","Data":"56c8fb9060f49e09f06691d7c914a0905cd0de9a4abcea04aa22a3f9209a1b62"}
Jul 24 14:32:24.463719 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:24.463663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fplzf" event={"ID":"7500633c-648e-4a58-bb81-54bd2d438612","Type":"ContainerStarted","Data":"5294a9f9cf05c7591a9e5dd95e130870fbb6e8fd788003046c11a390769d9088"}
Jul 24 14:32:24.467559 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:24.467531    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nzj76" event={"ID":"ade776d8-e297-4e5c-b57e-8e6cd942c159","Type":"ContainerStarted","Data":"7d66a7fa795ea85a5a64b8657a35284583f99cae6184fc14999d7d8a6792f4fb"}
Jul 24 14:32:24.471062 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:24.471036    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal" event={"ID":"346748fe11d636dec2275399a789dab0","Type":"ContainerStarted","Data":"6b245ba974cd083a9e7909eb10011b0bba81c545e49598b9a030d553cc4d4844"}
Jul 24 14:32:24.492844 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:24.492712    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal" podStartSLOduration=20.492692629 podStartE2EDuration="20.492692629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:24.49235314 +0000 UTC m=+22.996071100" watchObservedRunningTime="2026-07-24 14:32:24.492692629 +0000 UTC m=+22.996410591"
Jul 24 14:32:24.492844 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:24.492812    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6vwwl" podStartSLOduration=0.963705693 podStartE2EDuration="20.492806392s" totalImagesPullingTime="19.529100699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:24.474071821 +0000 UTC m=+22.977789783" watchObservedRunningTime="2026-07-24 14:32:24.492806392 +0000 UTC m=+22.996524351"
Jul 24 14:32:25.475231 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.474990    2581 generic.go:356] "Generic (PLEG): container finished" podID="905224f3fd7ab7503a387859310ffa2d" containerID="37cad679a6c04726087729941af8fd22bc22079687be76094bd4ce25278a3a80" exitCode=0
Jul 24 14:32:25.476069 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.475080    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" event={"ID":"905224f3fd7ab7503a387859310ffa2d","Type":"ContainerDied","Data":"37cad679a6c04726087729941af8fd22bc22079687be76094bd4ce25278a3a80"}
Jul 24 14:32:25.476881 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.476856    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fmzbh" event={"ID":"0d372f67-c893-46a9-8ce9-64507e3ae5ea","Type":"ContainerStarted","Data":"96652017d66dfce1a028140792c7fb8f9d0c40b919bfbe3fe7cce3e982ff1785"}
Jul 24 14:32:25.478573 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.478545    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g44tl" event={"ID":"7f98367c-0a6f-491a-a185-8f34de518278","Type":"ContainerStarted","Data":"7be2104564d21781cd5007186c82b294a1e1e64f869cd6261727b3b131289b8d"}
Jul 24 14:32:25.482002 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.481978    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg" event={"ID":"aa99a100-21cb-48e5-b249-2d3a951bb436","Type":"ContainerStarted","Data":"7fd0f94c7622eb85a9e82790dbc55919782b650db4210a4503142917d60855b1"}
Jul 24 14:32:25.482114 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.482052    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg" event={"ID":"aa99a100-21cb-48e5-b249-2d3a951bb436","Type":"ContainerStarted","Data":"c1eb6196cbb0414e8da01d99bd88d24e597bd2ce1f2ed3d8129a036df1bd2c53"}
Jul 24 14:32:25.482185 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.482117    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg" event={"ID":"aa99a100-21cb-48e5-b249-2d3a951bb436","Type":"ContainerStarted","Data":"1e52675268c9388e551e66e49488e000d4576c214746174786e2d5e19a0c9c2f"}
Jul 24 14:32:25.482185 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.482162    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg" event={"ID":"aa99a100-21cb-48e5-b249-2d3a951bb436","Type":"ContainerStarted","Data":"71df1b94491e4bc245a26fe8c1fa7fe226a62cd47d7127cff72c3055504c3560"}
Jul 24 14:32:25.482297 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.482217    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg" event={"ID":"aa99a100-21cb-48e5-b249-2d3a951bb436","Type":"ContainerStarted","Data":"efbe0b94c28ee54394fdcda9c479f7321b0801c78987954ed31afb8ed45dbf1e"}
Jul 24 14:32:25.482297 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.482264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg" event={"ID":"aa99a100-21cb-48e5-b249-2d3a951bb436","Type":"ContainerStarted","Data":"f0f6023f2b3f33f62abc30d70c11628957b54ff88a0637ea18a47c55fb3e165e"}
Jul 24 14:32:25.483755 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.483734    2581 generic.go:356] "Generic (PLEG): container finished" podID="c1aafd01-1d92-4f6e-b4d3-aceee2f33332" containerID="1a86a6afb0be2cb7725b961873f3522e035fdecbce90c025928095317aebe906" exitCode=0
Jul 24 14:32:25.483883 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.483812    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wh679" event={"ID":"c1aafd01-1d92-4f6e-b4d3-aceee2f33332","Type":"ContainerDied","Data":"1a86a6afb0be2cb7725b961873f3522e035fdecbce90c025928095317aebe906"}
Jul 24 14:32:25.485705 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.485680    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf" event={"ID":"c3fc1140-588d-41ec-a1d4-50822e44b72e","Type":"ContainerStarted","Data":"1f4b208f2ce6940a43aec37517e230e71bfce3bf5e7dcd339e5926a4dda93b92"}
Jul 24 14:32:25.487474 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.487448    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fplzf" event={"ID":"7500633c-648e-4a58-bb81-54bd2d438612","Type":"ContainerStarted","Data":"be81ad5e2ee467748db15b559ddba5b06ae69478971668fb2a3160d0d4a977d6"}
Jul 24 14:32:25.491831 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.491641    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nzj76" podStartSLOduration=1.9671849209999999 podStartE2EDuration="21.491623718s" totalImagesPullingTime="19.524438797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:24.514934415 +0000 UTC m=+23.018652379" watchObservedRunningTime="2026-07-24 14:32:25.491623718 +0000 UTC m=+23.995341681"
Jul 24 14:32:25.510748 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.510579    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fmzbh" podStartSLOduration=2.014952241 podStartE2EDuration="21.510562469s" totalImagesPullingTime="19.495610228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:25.510489028 +0000 UTC m=+24.014206986" watchObservedRunningTime="2026-07-24 14:32:25.510562469 +0000 UTC m=+24.014280426"
Jul 24 14:32:25.554166 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.554117    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g44tl" podStartSLOduration=2.043443823 podStartE2EDuration="21.554080186s" totalImagesPullingTime="19.510636363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:25.553407996 +0000 UTC m=+24.057125966" watchObservedRunningTime="2026-07-24 14:32:25.554080186 +0000 UTC m=+24.057798144"
Jul 24 14:32:25.569764 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.569728    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fplzf" podStartSLOduration=5.569713843 podStartE2EDuration="5.569713843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:25.569566866 +0000 UTC m=+24.073284829" watchObservedRunningTime="2026-07-24 14:32:25.569713843 +0000 UTC m=+24.073431801"
Jul 24 14:32:25.609367 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.609323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:25.689970 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:25.689930    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:26.179771 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.179743    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 14:32:26.265961 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.265777    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T14:32:26.179766014Z","UUID":"304e5512-2078-4768-9d48-d64c96ec4c38","Handler":null,"Name":"","Endpoint":""}
Jul 24 14:32:26.267617 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.267597    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 14:32:26.267751 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.267639    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 14:32:26.328252 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.328173    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:26.328450 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.328183    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:26.328450 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:26.328317    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:26.328450 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.328183    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:26.328450 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:26.328384    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:26.328633 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:26.328459    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:26.370107 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.370072    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:26.442594 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.442555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:26.491661 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.491631    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf" event={"ID":"c3fc1140-588d-41ec-a1d4-50822e44b72e","Type":"ContainerStarted","Data":"6358555cc9495c7838eeb78f2a81837f66f7ae68ae98bd42d83e9dc0a6179df8"}
Jul 24 14:32:26.493213 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.493169    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2f654" event={"ID":"c5b7db0b-7ea5-477f-b451-9cc880feec48","Type":"ContainerStarted","Data":"215bb2cebaaf6bcbe91caade4d55d8c7f8074cba5810260579fc23c9a96263c7"}
Jul 24 14:32:26.495195 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.495150    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" event={"ID":"905224f3fd7ab7503a387859310ffa2d","Type":"ContainerStarted","Data":"b802697a8a8c5a283262deacbd5c61f9eb499ae05a47e8b07d21bd74a4642969"}
Jul 24 14:32:26.510860 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.510811    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2f654" podStartSLOduration=2.911610434 podStartE2EDuration="22.510792632s" totalImagesPullingTime="19.599182198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:26.510668485 +0000 UTC m=+25.014386443" watchObservedRunningTime="2026-07-24 14:32:26.510792632 +0000 UTC m=+25.014510590"
Jul 24 14:32:26.528606 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.528561    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" podStartSLOduration=22.528545938 podStartE2EDuration="22.528545938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:26.528274533 +0000 UTC m=+25.031992495" watchObservedRunningTime="2026-07-24 14:32:26.528545938 +0000 UTC m=+25.032263895"
Jul 24 14:32:26.781188 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.781150    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:26.839726 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:26.839669    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:27.175719 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:27.175685    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:27.275650 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:27.275605    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:27.501934 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:27.501893    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg" event={"ID":"aa99a100-21cb-48e5-b249-2d3a951bb436","Type":"ContainerStarted","Data":"c9efd110bffa8c0d3d070cd766eca25822234ca5b650653a78f61db9039fde25"}
Jul 24 14:32:27.504363 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:27.504324    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf" event={"ID":"c3fc1140-588d-41ec-a1d4-50822e44b72e","Type":"ContainerStarted","Data":"b33e332ab070727e8209f676a8343c3a596c061bb4a20141402c5ad3a27b3a08"}
Jul 24 14:32:27.524721 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:27.524675    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-chbwf" podStartSLOduration=1.51819638 podStartE2EDuration="23.524660187s" totalImagesPullingTime="22.006463807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:27.524214115 +0000 UTC m=+26.027932074" watchObservedRunningTime="2026-07-24 14:32:27.524660187 +0000 UTC m=+26.028378167"
Jul 24 14:32:28.328005 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:28.327971    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:28.328005 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:28.327997    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:28.328304 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:28.328104    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:28.328304 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:28.328115    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:28.328304 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:28.328204    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:28.328304 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:28.328281    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:30.328400 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.328216    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:30.329078 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.328271    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:30.329078 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:30.328473    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:30.329078 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.328270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:30.329078 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:30.328563    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:30.329078 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:30.328633    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:30.443688 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.443661    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:30.444204 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.444188    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:30.516513 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.516475    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg" event={"ID":"aa99a100-21cb-48e5-b249-2d3a951bb436","Type":"ContainerStarted","Data":"be437bbc308f9e7effbe2e3ed60f713fd0598161cdd6e141f0b25a3bb7fbae28"}
Jul 24 14:32:30.516853 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.516815    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:30.516989 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.516892    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:30.516989 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.516941    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:30.518760 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.518738    2581 generic.go:356] "Generic (PLEG): container finished" podID="c1aafd01-1d92-4f6e-b4d3-aceee2f33332" containerID="234457ae63d5192d062563c72fc59d7df7740b678d18726e58a8f17d0184ad21" exitCode=0
Jul 24 14:32:30.518878 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.518815    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wh679" event={"ID":"c1aafd01-1d92-4f6e-b4d3-aceee2f33332","Type":"ContainerDied","Data":"234457ae63d5192d062563c72fc59d7df7740b678d18726e58a8f17d0184ad21"}
Jul 24 14:32:30.519129 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.519111    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:30.519907 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.519551    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g44tl"
Jul 24 14:32:30.534000 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.533974    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:30.537733 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.537707    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:32:30.539793 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.539763    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:30.546559 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.546521    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg" podStartSLOduration=6.724608129 podStartE2EDuration="26.546499266s" totalImagesPullingTime="19.821891137s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:30.546175468 +0000 UTC m=+29.049893430" watchObservedRunningTime="2026-07-24 14:32:30.546499266 +0000 UTC m=+29.050217202"
Jul 24 14:32:30.609844 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.609807    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:30.672738 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.672706    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:30.905879 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.905796    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:30.969693 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:30.969666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:31.623575 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:31.623354    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-snn2w"]
Jul 24 14:32:31.624030 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:31.623671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:31.624030 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:31.623756    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:31.627505 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:31.627480    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wxpsq"]
Jul 24 14:32:31.627681 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:31.627625    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:31.627750 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:31.627713    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:31.628181 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:31.628162    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vqpl5"]
Jul 24 14:32:31.628292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:31.628262    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:31.628355 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:31.628338    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:31.952898 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:31.952865    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:31.962462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:31.962441    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fplzf_7500633c-648e-4a58-bb81-54bd2d438612/dns-node-resolver/0.log"
Jul 24 14:32:32.044074 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:32.044043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:32.144996 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:32.144909    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:32.368833 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:32.368796    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:32.434987 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:32.434920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:32.525785 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:32.525753    2581 generic.go:356] "Generic (PLEG): container finished" podID="c1aafd01-1d92-4f6e-b4d3-aceee2f33332" containerID="498d020c163596211cdb01649402243a3be2f505720565fac1256f1f387ee1c3" exitCode=0
Jul 24 14:32:32.525934 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:32.525816    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wh679" event={"ID":"c1aafd01-1d92-4f6e-b4d3-aceee2f33332","Type":"ContainerDied","Data":"498d020c163596211cdb01649402243a3be2f505720565fac1256f1f387ee1c3"}
Jul 24 14:32:33.065801 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:33.065772    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:33.156524 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:33.156497    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fmzbh_0d372f67-c893-46a9-8ce9-64507e3ae5ea/node-ca/0.log"
Jul 24 14:32:33.166984 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:33.166947    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:33.329406 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:33.329329    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:33.329583 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:33.329455    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:33.329583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:33.329470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:33.329583 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:33.329561    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:33.329755 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:33.329626    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:33.329755 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:33.329729    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:34.500240 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:34.500202    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:34.532173 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:34.532138    2581 generic.go:356] "Generic (PLEG): container finished" podID="c1aafd01-1d92-4f6e-b4d3-aceee2f33332" containerID="c46237c1287f7190904f6dd4c833a6105c589ab7a1f664876d2d0bc0ad789499" exitCode=0
Jul 24 14:32:34.532330 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:34.532190    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wh679" event={"ID":"c1aafd01-1d92-4f6e-b4d3-aceee2f33332","Type":"ContainerDied","Data":"c46237c1287f7190904f6dd4c833a6105c589ab7a1f664876d2d0bc0ad789499"}
Jul 24 14:32:34.705188 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:34.705150    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:34.802716 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:34.802637    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:35.328993 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:35.328957    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:35.328993 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:35.328981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:35.329281 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:35.329107    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:35.329281 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:35.329202    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:35.329281 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:35.329203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:35.329422 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:35.329288    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:35.705062 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:35.705025    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:35.957288 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:35.957193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:35.957288 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:35.957271    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:35.957497 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:35.957357    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:35.957497 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:35.957390    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:35.957497 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:35.957437    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs podName:0e4ffaf2-3ff8-4577-a883-956efc5f007a nodeName:}" failed. No retries permitted until 2026-07-24 14:33:07.957418528 +0000 UTC m=+66.461136476 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs") pod "network-metrics-daemon-wxpsq" (UID: "0e4ffaf2-3ff8-4577-a883-956efc5f007a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:32:35.957497 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:35.957461    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret podName:8f983040-8d3c-4f7c-8c82-12586a4033ee nodeName:}" failed. No retries permitted until 2026-07-24 14:33:07.957452508 +0000 UTC m=+66.461170451 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret") pod "global-pull-secret-syncer-snn2w" (UID: "8f983040-8d3c-4f7c-8c82-12586a4033ee") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:32:36.058511 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:36.058472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6tpg\" (UniqueName: \"kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg\") pod \"network-check-target-vqpl5\" (UID: \"5ed878d7-e106-4e24-a24b-46253570f910\") " pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:36.058681 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:36.058622    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:32:36.058681 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:36.058644    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:32:36.058681 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:36.058654    2581 projected.go:196] Error preparing data for projected volume kube-api-access-b6tpg for pod openshift-network-diagnostics/network-check-target-vqpl5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:36.058816 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:36.058714    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg podName:5ed878d7-e106-4e24-a24b-46253570f910 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:08.058696288 +0000 UTC m=+66.562414244 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6tpg" (UniqueName: "kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg") pod "network-check-target-vqpl5" (UID: "5ed878d7-e106-4e24-a24b-46253570f910") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:32:36.539691 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:36.539659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:36.698317 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:36.698279    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:36.799180 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:36.799079    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:36.872433 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:36.872395    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:36.965578 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:36.965535    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:37.328328 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:37.328295    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:37.328514 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:37.328295    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:37.328514 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:37.328432    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:37.328514 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:37.328295    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:37.328665 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:37.328510    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:37.328665 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:37.328572    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:37.431801 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:37.431766    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:37.823087 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:37.823038    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:38.278312 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:38.278233    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:38.626526 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:38.626297    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:39.234065 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:39.234030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:39.328605 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:39.328565    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:39.328773 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:39.328576    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:39.328773 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:39.328698    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:39.328773 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:39.328576    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:39.328773 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:39.328762    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:39.328939 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:39.328823    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:39.546821 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:39.546735    2581 generic.go:356] "Generic (PLEG): container finished" podID="c1aafd01-1d92-4f6e-b4d3-aceee2f33332" containerID="d62c2d1bc33b1793dc61b81e0f2d7febca9b88239237aeb199366a78fa6a17f3" exitCode=0
Jul 24 14:32:39.546821 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:39.546786    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wh679" event={"ID":"c1aafd01-1d92-4f6e-b4d3-aceee2f33332","Type":"ContainerDied","Data":"d62c2d1bc33b1793dc61b81e0f2d7febca9b88239237aeb199366a78fa6a17f3"}
Jul 24 14:32:39.557957 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:39.557933    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:39.839445 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:39.839365    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:40.054284 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:40.054257    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:40.240604 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:40.240569    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:40.332397 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:40.332358    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:40.552277 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:40.552195    2581 generic.go:356] "Generic (PLEG): container finished" podID="c1aafd01-1d92-4f6e-b4d3-aceee2f33332" containerID="fc99a5e3daeadf555fa21455bdacd680ca29482bedc63a6bff56605e6c70f7a8" exitCode=0
Jul 24 14:32:40.552277 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:40.552239    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wh679" event={"ID":"c1aafd01-1d92-4f6e-b4d3-aceee2f33332","Type":"ContainerDied","Data":"fc99a5e3daeadf555fa21455bdacd680ca29482bedc63a6bff56605e6c70f7a8"}
Jul 24 14:32:40.583388 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:40.583299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:40.680287 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:40.680259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:41.226150 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:41.226117    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:41.326426 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:41.326394    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:41.328233 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:41.328207    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:41.328233 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:41.328221    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:41.328614 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:41.328238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:41.328614 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:41.328321    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:41.328614 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:41.328398    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:41.328614 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:41.328477    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:41.392449 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:41.392400    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:41.558029 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:41.557947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wh679" event={"ID":"c1aafd01-1d92-4f6e-b4d3-aceee2f33332","Type":"ContainerStarted","Data":"d589cf03dbbcbba389933f72f352a3a0c32972dd81c0733134611ebccbb6fcd3"}
Jul 24 14:32:41.579413 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:41.579362    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wh679" podStartSLOduration=7.259499793 podStartE2EDuration="37.579344992s" totalImagesPullingTime="30.319845199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:41.578881859 +0000 UTC m=+40.082599819" watchObservedRunningTime="2026-07-24 14:32:41.579344992 +0000 UTC m=+40.083062952"
Jul 24 14:32:41.622273 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:41.622239    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:41.835669 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:41.835591    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:41.934143 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:41.934083    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:42.033544 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:42.033500    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:42.226187 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:42.226152    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:42.515842 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:42.515750    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:42.840419 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:42.840333    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:43.329412 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:43.329380    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:43.329606 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:43.329492    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:43.329606 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:43.329511    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:43.329606 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:43.329491    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:43.329606 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:43.329586    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:43.329821 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:43.329647    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:43.425244 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:43.425208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:43.828271 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:43.828233    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:44.022704 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:44.022670    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:44.220798 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:44.220758    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:44.624934 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:44.624899    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:45.221480 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:45.221449    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:45.328265 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:45.328225    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:45.328265 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:45.328261    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:45.328468 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:45.328354    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:45.328468 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:45.328439    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:45.328468 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:45.328434    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:45.328588 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:45.328532    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:45.631030 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:45.631003    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:45.705563 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:45.705530    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:45.832525 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:45.832490    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:46.026306 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:46.026223    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:46.431317 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:46.431283    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:46.626399 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:46.626364    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:46.827588 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:46.827501    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:47.232016 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:47.231979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:47.328684 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:47.328652    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:47.328840 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:47.328652    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:47.328840 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:47.328772    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:47.328909 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:47.328849    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:47.328909 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:47.328892    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:47.328975 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:47.328937    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:47.671142 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:47.671106    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:49.328918 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:49.328878    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:49.329420 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:49.328878    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:49.329420 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:49.329026    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:49.329420 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:49.329025    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:49.329420 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:49.329135    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:49.329420 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:49.329175    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:51.328926 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:51.328893    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:51.329444 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:51.328893    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:51.329444 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:51.329004    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:51.329444 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:51.329116    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:51.329444 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:51.328893    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:51.329444 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:51.329199    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:53.328746 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:53.328707    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:53.329190 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:53.328816    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:53.329190 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:53.328818    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:53.329190 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:53.328848    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:53.329190 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:53.328987    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:53.329190 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:53.329050    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:55.179225 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:55.179184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:55.297879 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:55.297839    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:55.328667 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:55.328641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:55.328837 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:55.328641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:55.328837 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:55.328753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:55.328837 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:55.328745    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-snn2w" podUID="8f983040-8d3c-4f7c-8c82-12586a4033ee"
Jul 24 14:32:55.328982 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:55.328827    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxpsq" podUID="0e4ffaf2-3ff8-4577-a883-956efc5f007a"
Jul 24 14:32:55.328982 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:32:55.328906    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqpl5" podUID="5ed878d7-e106-4e24-a24b-46253570f910"
Jul 24 14:32:55.489751 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:55.489660    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:55.606842 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:55.606813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:55.754227 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:55.754131    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:55.884402 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:55.884367    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:32:56.342570 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.342543    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeReady"
Jul 24 14:32:56.342956 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.342657    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 14:32:56.416233 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.416201    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cq9pm"]
Jul 24 14:32:56.429635 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.429604    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r5znj"]
Jul 24 14:32:56.429773 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.429751    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cq9pm"
Jul 24 14:32:56.434818 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.434794    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lb5b6\""
Jul 24 14:32:56.434951 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.434794    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 14:32:56.434951 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.434792    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 14:32:56.434951 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.434910    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 14:32:56.446408 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.446384    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.450569 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.450541    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 14:32:56.453204 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.453175    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m7vj4\""
Jul 24 14:32:56.453329 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.453210    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 14:32:56.453329 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.453236    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 14:32:56.453440 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.453334    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cq9pm"]
Jul 24 14:32:56.453492 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.453476    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 14:32:56.453979 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.453960    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r5znj"]
Jul 24 14:32:56.469666 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.469643    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fxnnt"]
Jul 24 14:32:56.485216 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.485194    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.488117 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.488084    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 14:32:56.488303 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.488242    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m2jtv\""
Jul 24 14:32:56.488557 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.488542    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 14:32:56.488762 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.488745    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fxnnt"]
Jul 24 14:32:56.510771 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.510740    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.510918 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.510792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-data-volume\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.510918 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.510818    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbqzm\" (UniqueName: \"kubernetes.io/projected/5e6f93bf-25de-425e-b119-96cbf90b4075-kube-api-access-xbqzm\") pod \"ingress-canary-cq9pm\" (UID: \"5e6f93bf-25de-425e-b119-96cbf90b4075\") " pod="openshift-ingress-canary/ingress-canary-cq9pm"
Jul 24 14:32:56.510918 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.510839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80e703d4-7f37-4591-9528-1a53effbe5c7-metrics-tls\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.510918 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.510883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80e703d4-7f37-4591-9528-1a53effbe5c7-tmp-dir\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.511149 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.510932    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.511149 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.510955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l46qd\" (UniqueName: \"kubernetes.io/projected/80e703d4-7f37-4591-9528-1a53effbe5c7-kube-api-access-l46qd\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.511149 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.511008    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e6f93bf-25de-425e-b119-96cbf90b4075-cert\") pod \"ingress-canary-cq9pm\" (UID: \"5e6f93bf-25de-425e-b119-96cbf90b4075\") " pod="openshift-ingress-canary/ingress-canary-cq9pm"
Jul 24 14:32:56.511149 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.511035    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-crio-socket\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.511149 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.511083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2642b\" (UniqueName: \"kubernetes.io/projected/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-kube-api-access-2642b\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.511149 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.511141    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80e703d4-7f37-4591-9528-1a53effbe5c7-config-volume\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.611693 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.611664    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e6f93bf-25de-425e-b119-96cbf90b4075-cert\") pod \"ingress-canary-cq9pm\" (UID: \"5e6f93bf-25de-425e-b119-96cbf90b4075\") " pod="openshift-ingress-canary/ingress-canary-cq9pm"
Jul 24 14:32:56.611848 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.611700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-crio-socket\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.611848 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.611733    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2642b\" (UniqueName: \"kubernetes.io/projected/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-kube-api-access-2642b\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.611933 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.611851    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80e703d4-7f37-4591-9528-1a53effbe5c7-config-volume\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.611933 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.611887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-crio-socket\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.611933 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.611890    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.612060 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.611956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-data-volume\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.612060 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.611982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbqzm\" (UniqueName: \"kubernetes.io/projected/5e6f93bf-25de-425e-b119-96cbf90b4075-kube-api-access-xbqzm\") pod \"ingress-canary-cq9pm\" (UID: \"5e6f93bf-25de-425e-b119-96cbf90b4075\") " pod="openshift-ingress-canary/ingress-canary-cq9pm"
Jul 24 14:32:56.612060 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.612040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80e703d4-7f37-4591-9528-1a53effbe5c7-metrics-tls\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.612237 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.612073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80e703d4-7f37-4591-9528-1a53effbe5c7-tmp-dir\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.612237 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.612134    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.612237 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.612157    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l46qd\" (UniqueName: \"kubernetes.io/projected/80e703d4-7f37-4591-9528-1a53effbe5c7-kube-api-access-l46qd\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.612439 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.612327    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-data-volume\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.612547 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.612511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80e703d4-7f37-4591-9528-1a53effbe5c7-config-volume\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.612547 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.612543    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80e703d4-7f37-4591-9528-1a53effbe5c7-tmp-dir\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.612803 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.612782    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.615918 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.615894    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80e703d4-7f37-4591-9528-1a53effbe5c7-metrics-tls\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.615918 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.615920    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e6f93bf-25de-425e-b119-96cbf90b4075-cert\") pod \"ingress-canary-cq9pm\" (UID: \"5e6f93bf-25de-425e-b119-96cbf90b4075\") " pod="openshift-ingress-canary/ingress-canary-cq9pm"
Jul 24 14:32:56.616052 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.615892    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.621336 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.621307    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l46qd\" (UniqueName: \"kubernetes.io/projected/80e703d4-7f37-4591-9528-1a53effbe5c7-kube-api-access-l46qd\") pod \"dns-default-fxnnt\" (UID: \"80e703d4-7f37-4591-9528-1a53effbe5c7\") " pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.621336 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.621325    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2642b\" (UniqueName: \"kubernetes.io/projected/d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f-kube-api-access-2642b\") pod \"insights-runtime-extractor-r5znj\" (UID: \"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f\") " pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.621534 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.621514    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbqzm\" (UniqueName: \"kubernetes.io/projected/5e6f93bf-25de-425e-b119-96cbf90b4075-kube-api-access-xbqzm\") pod \"ingress-canary-cq9pm\" (UID: \"5e6f93bf-25de-425e-b119-96cbf90b4075\") " pod="openshift-ingress-canary/ingress-canary-cq9pm"
Jul 24 14:32:56.738668 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.738628    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cq9pm"
Jul 24 14:32:56.758308 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.758281    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r5znj"
Jul 24 14:32:56.795435 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.795405    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:32:56.904310 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.904195    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cq9pm"]
Jul 24 14:32:56.909063 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:56.909020    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6f93bf_25de_425e_b119_96cbf90b4075.slice/crio-d92fd4127798e20de91f12b391295ec7276c91215305e313f714311d210fd8a6 WatchSource:0}: Error finding container d92fd4127798e20de91f12b391295ec7276c91215305e313f714311d210fd8a6: Status 404 returned error can't find the container with id d92fd4127798e20de91f12b391295ec7276c91215305e313f714311d210fd8a6
Jul 24 14:32:56.922625 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.922557    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r5znj"]
Jul 24 14:32:56.927164 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:56.927133    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd88021fe_bdc7_4953_8bb5_eeeb8bb33b8f.slice/crio-65c172543349b886a58c60e6a6373a6a2efc95a4d1785cf370b2ef1e5cf10f66 WatchSource:0}: Error finding container 65c172543349b886a58c60e6a6373a6a2efc95a4d1785cf370b2ef1e5cf10f66: Status 404 returned error can't find the container with id 65c172543349b886a58c60e6a6373a6a2efc95a4d1785cf370b2ef1e5cf10f66
Jul 24 14:32:56.958179 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:56.958148    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fxnnt"]
Jul 24 14:32:56.961248 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:32:56.961219    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80e703d4_7f37_4591_9528_1a53effbe5c7.slice/crio-eb9819d6f27f49bf25eba26187d2d85f0d80cc2b77b3b671b7348966eeb46797 WatchSource:0}: Error finding container eb9819d6f27f49bf25eba26187d2d85f0d80cc2b77b3b671b7348966eeb46797: Status 404 returned error can't find the container with id eb9819d6f27f49bf25eba26187d2d85f0d80cc2b77b3b671b7348966eeb46797
Jul 24 14:32:57.328621 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.328505    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:32:57.328782 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.328505    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:32:57.329029 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.328905    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:32:57.332626 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.332600    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 14:32:57.332998 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.332866    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ljg7j\""
Jul 24 14:32:57.333857 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.333629    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 14:32:57.333857 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.333661    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tpbrb\""
Jul 24 14:32:57.333857 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.333728    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 14:32:57.334295 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.334220    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 14:32:57.599955 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.599785    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fxnnt" event={"ID":"80e703d4-7f37-4591-9528-1a53effbe5c7","Type":"ContainerStarted","Data":"eb9819d6f27f49bf25eba26187d2d85f0d80cc2b77b3b671b7348966eeb46797"}
Jul 24 14:32:57.602520 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.602285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5znj" event={"ID":"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f","Type":"ContainerStarted","Data":"65c172543349b886a58c60e6a6373a6a2efc95a4d1785cf370b2ef1e5cf10f66"}
Jul 24 14:32:57.604736 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:32:57.604698    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cq9pm" event={"ID":"5e6f93bf-25de-425e-b119-96cbf90b4075","Type":"ContainerStarted","Data":"d92fd4127798e20de91f12b391295ec7276c91215305e313f714311d210fd8a6"}
Jul 24 14:33:00.674478 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:00.674433    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:00.779461 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:00.779418    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:00.879286 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:00.879255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:00.881326 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:00.881295    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"]
Jul 24 14:33:00.894794 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:00.894762    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"
Jul 24 14:33:00.896577 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:00.896548    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"]
Jul 24 14:33:00.897555 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:00.897530    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-ddj5k\""
Jul 24 14:33:00.897681 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:00.897530    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 24 14:33:00.944742 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:00.944658    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dbf39a86-7562-46ec-97a7-2b01531d9b2d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-94nkg\" (UID: \"dbf39a86-7562-46ec-97a7-2b01531d9b2d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"
Jul 24 14:33:01.045900 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:01.045870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dbf39a86-7562-46ec-97a7-2b01531d9b2d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-94nkg\" (UID: \"dbf39a86-7562-46ec-97a7-2b01531d9b2d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"
Jul 24 14:33:01.046082 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:33:01.046008    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 14:33:01.046082 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:33:01.046080    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf39a86-7562-46ec-97a7-2b01531d9b2d-tls-certificates podName:dbf39a86-7562-46ec-97a7-2b01531d9b2d nodeName:}" failed. No retries permitted until 2026-07-24 14:33:01.546059182 +0000 UTC m=+60.049777136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/dbf39a86-7562-46ec-97a7-2b01531d9b2d-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-94nkg" (UID: "dbf39a86-7562-46ec-97a7-2b01531d9b2d") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 14:33:01.264680 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:01.264551    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:01.357996 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:01.357934    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:01.430636 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:01.430579    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:01.549210 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:01.549128    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dbf39a86-7562-46ec-97a7-2b01531d9b2d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-94nkg\" (UID: \"dbf39a86-7562-46ec-97a7-2b01531d9b2d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"
Jul 24 14:33:01.549359 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:33:01.549302    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 14:33:01.549411 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:33:01.549384    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf39a86-7562-46ec-97a7-2b01531d9b2d-tls-certificates podName:dbf39a86-7562-46ec-97a7-2b01531d9b2d nodeName:}" failed. No retries permitted until 2026-07-24 14:33:02.549365424 +0000 UTC m=+61.053083380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/dbf39a86-7562-46ec-97a7-2b01531d9b2d-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-94nkg" (UID: "dbf39a86-7562-46ec-97a7-2b01531d9b2d") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 24 14:33:01.714694 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:01.714625    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:01.813081 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:01.813001    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:01.926733 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:01.926698    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:02.022737 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:02.022686    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:02.541633 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:02.541608    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4zgrg"
Jul 24 14:33:02.556877 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:02.556839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dbf39a86-7562-46ec-97a7-2b01531d9b2d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-94nkg\" (UID: \"dbf39a86-7562-46ec-97a7-2b01531d9b2d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"
Jul 24 14:33:02.559289 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:02.559260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/dbf39a86-7562-46ec-97a7-2b01531d9b2d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-94nkg\" (UID: \"dbf39a86-7562-46ec-97a7-2b01531d9b2d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"
Jul 24 14:33:02.707246 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:02.707219    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"
Jul 24 14:33:02.720750 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:02.720722    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:02.813992 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:02.813958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:02.918962 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:02.918897    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:02.933334 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:02.933306    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"]
Jul 24 14:33:02.936972 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:33:02.936948    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf39a86_7562_46ec_97a7_2b01531d9b2d.slice/crio-a0de4dcae65834abef958b1324f356260760523f045f992ba2658221d0a9758e WatchSource:0}: Error finding container a0de4dcae65834abef958b1324f356260760523f045f992ba2658221d0a9758e: Status 404 returned error can't find the container with id a0de4dcae65834abef958b1324f356260760523f045f992ba2658221d0a9758e
Jul 24 14:33:03.023503 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.023464    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:03.126221 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.126188    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:03.202132 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.202077    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:03.352572 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.352531    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:03.468706 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.468611    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:03.565267 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.565214    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:03.625036 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.624997    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg" event={"ID":"dbf39a86-7562-46ec-97a7-2b01531d9b2d","Type":"ContainerStarted","Data":"a0de4dcae65834abef958b1324f356260760523f045f992ba2658221d0a9758e"}
Jul 24 14:33:03.626925 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.626890    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fxnnt" event={"ID":"80e703d4-7f37-4591-9528-1a53effbe5c7","Type":"ContainerStarted","Data":"58a7e9e947d5843ccc4be321bd135be4649bc5d8cb83d5b86f16c5ef4aca27f6"}
Jul 24 14:33:03.627070 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.626947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fxnnt" event={"ID":"80e703d4-7f37-4591-9528-1a53effbe5c7","Type":"ContainerStarted","Data":"202b57427c7f0b2f15219276cad589f7b3df0933b3d56da32b7f96bcd59593eb"}
Jul 24 14:33:03.627070 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.627018    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:33:03.628667 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.628626    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5znj" event={"ID":"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f","Type":"ContainerStarted","Data":"106c2d15dcef2fdceff01ea2a54238c9f9046e6338503b441cd687525e413685"}
Jul 24 14:33:03.630211 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.630189    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cq9pm" event={"ID":"5e6f93bf-25de-425e-b119-96cbf90b4075","Type":"ContainerStarted","Data":"c609ef9669d2c52cb712936ca53b10f537edd0415fc3708a713ebad657094c2c"}
Jul 24 14:33:03.657946 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:03.657890    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fxnnt" podStartSLOduration=2.043922817 podStartE2EDuration="7.657868406s" totalImagesPullingTime="5.613945589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:03.65732245 +0000 UTC m=+62.161040406" watchObservedRunningTime="2026-07-24 14:33:03.657868406 +0000 UTC m=+62.161586365"
Jul 24 14:33:07.647751 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.647706    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5znj" event={"ID":"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f","Type":"ContainerStarted","Data":"6b3ec0b62e49646b4218c4169b5ef630d61c24121272bd227fa8613edf95ca18"}
Jul 24 14:33:07.649438 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.649410    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg" event={"ID":"dbf39a86-7562-46ec-97a7-2b01531d9b2d","Type":"ContainerStarted","Data":"a94d57d2a790a599c078a4ef9cdf7a59c351bec01bf0fa35f07911f3b373a62b"}
Jul 24 14:33:07.649699 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.649674    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"
Jul 24 14:33:07.655131 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.655111    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg"
Jul 24 14:33:07.667669 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.667615    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-94nkg" podStartSLOduration=5.904016792 podStartE2EDuration="7.667597002s" totalImagesPullingTime="1.76358021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:07.667193591 +0000 UTC m=+66.170911563" watchObservedRunningTime="2026-07-24 14:33:07.667597002 +0000 UTC m=+66.171314963"
Jul 24 14:33:07.667833 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.667805    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cq9pm" podStartSLOduration=6.001989972 podStartE2EDuration="11.66779571s" totalImagesPullingTime="5.665805738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:03.677290229 +0000 UTC m=+62.181008188" watchObservedRunningTime="2026-07-24 14:33:07.66779571 +0000 UTC m=+66.171513669"
Jul 24 14:33:07.901259 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.901162    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:07.940841 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.940806    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"]
Jul 24 14:33:07.942749 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.942724    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:07.945468 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.945443    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 14:33:07.945609 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.945484    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 24 14:33:07.945609 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.945507    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 14:33:07.946338 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.946318    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 14:33:07.946569 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.946553    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ssbqq\""
Jul 24 14:33:07.946569 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.946562    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 24 14:33:07.953272 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.953251    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"]
Jul 24 14:33:07.981244 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:07.981208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:08.008229 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.008184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3d13857-8b7f-4996-9041-5fd1f8f84111-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.008389 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.008236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:33:08.008389 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.008302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3d13857-8b7f-4996-9041-5fd1f8f84111-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.008389 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.008327    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncrpt\" (UniqueName: \"kubernetes.io/projected/e3d13857-8b7f-4996-9041-5fd1f8f84111-kube-api-access-ncrpt\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.008389 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.008370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:33:08.008618 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.008413    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3d13857-8b7f-4996-9041-5fd1f8f84111-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.011041 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.011010    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 14:33:08.011041 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.011032    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 14:33:08.021549 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.021513    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e4ffaf2-3ff8-4577-a883-956efc5f007a-metrics-certs\") pod \"network-metrics-daemon-wxpsq\" (UID: \"0e4ffaf2-3ff8-4577-a883-956efc5f007a\") " pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:33:08.021673 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.021555    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f983040-8d3c-4f7c-8c82-12586a4033ee-original-pull-secret\") pod \"global-pull-secret-syncer-snn2w\" (UID: \"8f983040-8d3c-4f7c-8c82-12586a4033ee\") " pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:33:08.108950 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.108917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3d13857-8b7f-4996-9041-5fd1f8f84111-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.108950 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.108952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncrpt\" (UniqueName: \"kubernetes.io/projected/e3d13857-8b7f-4996-9041-5fd1f8f84111-kube-api-access-ncrpt\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.109220 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.108985    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3d13857-8b7f-4996-9041-5fd1f8f84111-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.109220 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.109023    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6tpg\" (UniqueName: \"kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg\") pod \"network-check-target-vqpl5\" (UID: \"5ed878d7-e106-4e24-a24b-46253570f910\") " pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:33:08.109220 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.109060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3d13857-8b7f-4996-9041-5fd1f8f84111-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.109366 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:33:08.109272    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 24 14:33:08.109366 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:33:08.109336    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3d13857-8b7f-4996-9041-5fd1f8f84111-prometheus-operator-tls podName:e3d13857-8b7f-4996-9041-5fd1f8f84111 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:08.60931695 +0000 UTC m=+67.113034900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/e3d13857-8b7f-4996-9041-5fd1f8f84111-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-xhlxv" (UID: "e3d13857-8b7f-4996-9041-5fd1f8f84111") : secret "prometheus-operator-tls" not found
Jul 24 14:33:08.109784 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.109754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3d13857-8b7f-4996-9041-5fd1f8f84111-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.111823 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.111796    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3d13857-8b7f-4996-9041-5fd1f8f84111-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.111940 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.111870    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 14:33:08.119495 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.119468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncrpt\" (UniqueName: \"kubernetes.io/projected/e3d13857-8b7f-4996-9041-5fd1f8f84111-kube-api-access-ncrpt\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.122029 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.121993    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 14:33:08.132867 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.132842    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6tpg\" (UniqueName: \"kubernetes.io/projected/5ed878d7-e106-4e24-a24b-46253570f910-kube-api-access-b6tpg\") pod \"network-check-target-vqpl5\" (UID: \"5ed878d7-e106-4e24-a24b-46253570f910\") " pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:33:08.146361 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.146340    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ljg7j\""
Jul 24 14:33:08.153725 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.153658    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-snn2w"
Jul 24 14:33:08.153842 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.153657    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:33:08.164663 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.164630    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tpbrb\""
Jul 24 14:33:08.172144 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.172107    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxpsq"
Jul 24 14:33:08.320369 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.320337    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-snn2w"]
Jul 24 14:33:08.344191 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.344161    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vqpl5"]
Jul 24 14:33:08.346008 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.345673    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:08.347485 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:33:08.347453    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ed878d7_e106_4e24_a24b_46253570f910.slice/crio-6cb756c5d4ef1e1b6a07f1a42b4795daa068125a5e54031e5cd29217e5a86f58 WatchSource:0}: Error finding container 6cb756c5d4ef1e1b6a07f1a42b4795daa068125a5e54031e5cd29217e5a86f58: Status 404 returned error can't find the container with id 6cb756c5d4ef1e1b6a07f1a42b4795daa068125a5e54031e5cd29217e5a86f58
Jul 24 14:33:08.364013 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.363982    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wxpsq"]
Jul 24 14:33:08.367208 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:33:08.367179    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e4ffaf2_3ff8_4577_a883_956efc5f007a.slice/crio-5b9a6fd8f224b0e70d8a7b41491a80930a03b3643ff375d45aa2759adcd13e99 WatchSource:0}: Error finding container 5b9a6fd8f224b0e70d8a7b41491a80930a03b3643ff375d45aa2759adcd13e99: Status 404 returned error can't find the container with id 5b9a6fd8f224b0e70d8a7b41491a80930a03b3643ff375d45aa2759adcd13e99
Jul 24 14:33:08.613718 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.613681    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3d13857-8b7f-4996-9041-5fd1f8f84111-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.616321 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.616284    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e3d13857-8b7f-4996-9041-5fd1f8f84111-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-xhlxv\" (UID: \"e3d13857-8b7f-4996-9041-5fd1f8f84111\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:08.654140 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.654105    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxpsq" event={"ID":"0e4ffaf2-3ff8-4577-a883-956efc5f007a","Type":"ContainerStarted","Data":"5b9a6fd8f224b0e70d8a7b41491a80930a03b3643ff375d45aa2759adcd13e99"}
Jul 24 14:33:08.655370 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.655336    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vqpl5" event={"ID":"5ed878d7-e106-4e24-a24b-46253570f910","Type":"ContainerStarted","Data":"6cb756c5d4ef1e1b6a07f1a42b4795daa068125a5e54031e5cd29217e5a86f58"}
Jul 24 14:33:08.656671 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.656641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-snn2w" event={"ID":"8f983040-8d3c-4f7c-8c82-12586a4033ee","Type":"ContainerStarted","Data":"3a930a6d295f28c56020b77431061813c07369f4b1f65cb413f6fc37f3fca0b6"}
Jul 24 14:33:08.854703 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:08.854667    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"
Jul 24 14:33:09.752345 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:09.752312    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:09.814188 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:09.814150    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:10.575344 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:10.575310    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-xhlxv"]
Jul 24 14:33:10.578737 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:33:10.578710    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d13857_8b7f_4996_9041_5fd1f8f84111.slice/crio-203ebaa779938db746953a3f0b887f5a6763f41f465663c5ce5b306cc07c7866 WatchSource:0}: Error finding container 203ebaa779938db746953a3f0b887f5a6763f41f465663c5ce5b306cc07c7866: Status 404 returned error can't find the container with id 203ebaa779938db746953a3f0b887f5a6763f41f465663c5ce5b306cc07c7866
Jul 24 14:33:10.665604 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:10.665575    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv" event={"ID":"e3d13857-8b7f-4996-9041-5fd1f8f84111","Type":"ContainerStarted","Data":"203ebaa779938db746953a3f0b887f5a6763f41f465663c5ce5b306cc07c7866"}
Jul 24 14:33:11.644821 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:11.644779    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:11.671654 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:11.671611    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5znj" event={"ID":"d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f","Type":"ContainerStarted","Data":"4d2108be85695ab880a48cb3d62a610a3d5715b4e4d274a8128fa4768b9e4b85"}
Jul 24 14:33:11.675014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:11.674871    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxpsq" event={"ID":"0e4ffaf2-3ff8-4577-a883-956efc5f007a","Type":"ContainerStarted","Data":"22348fcdf00029e9c9fdab6df279852cd403c7aa7c8c51c014558a8133825851"}
Jul 24 14:33:11.675014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:11.674928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxpsq" event={"ID":"0e4ffaf2-3ff8-4577-a883-956efc5f007a","Type":"ContainerStarted","Data":"06d2e8cfca5557979b739110785bd0ffa0e941044345ffd641f3e812707ba915"}
Jul 24 14:33:11.692460 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:11.691987    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r5znj" podStartSLOduration=1.957520328 podStartE2EDuration="15.691966638s" totalImagesPullingTime="13.73444631s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:11.690224125 +0000 UTC m=+70.193942084" watchObservedRunningTime="2026-07-24 14:33:11.691966638 +0000 UTC m=+70.195684597"
Jul 24 14:33:11.708530 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:11.708469    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wxpsq" podStartSLOduration=64.713754815 podStartE2EDuration="1m7.708446635s" totalImagesPullingTime="2.99469182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:11.706780582 +0000 UTC m=+70.210498541" watchObservedRunningTime="2026-07-24 14:33:11.708446635 +0000 UTC m=+70.212164594"
Jul 24 14:33:11.717057 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:11.717021    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:11.805875 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:11.805825    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:11.905592 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:11.905470    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:12.499905 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:12.499811    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:12.601194 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:12.601142    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:12.681695 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:12.681652    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv" event={"ID":"e3d13857-8b7f-4996-9041-5fd1f8f84111","Type":"ContainerStarted","Data":"84c3f0be2606349489c4ed9b2b859fea3e1d8c888f8c09924b10fdb25dce33b9"}
Jul 24 14:33:12.682199 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:12.681716    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv" event={"ID":"e3d13857-8b7f-4996-9041-5fd1f8f84111","Type":"ContainerStarted","Data":"d1abdd0905d234f1662910a5d26f9944df6b6278a02a2702ff19b4af482bfb64"}
Jul 24 14:33:12.701704 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:12.701365    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-xhlxv" podStartSLOduration=4.088678679 podStartE2EDuration="5.7013418s" totalImagesPullingTime="1.612663121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:12.700239559 +0000 UTC m=+71.203957514" watchObservedRunningTime="2026-07-24 14:33:12.7013418 +0000 UTC m=+71.205059760"
Jul 24 14:33:12.702389 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:12.702316    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:13.635695 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:13.635655    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fxnnt"
Jul 24 14:33:13.771654 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:13.771607    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:13.828252 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:13.828205    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:14.044793 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.044707    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:14.198002 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.197963    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:14.278560 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.278514    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:14.320354 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.320278    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xbksj"]
Jul 24 14:33:14.322816 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.322792    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.326760 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.326730    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 14:33:14.327123 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.327086    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 14:33:14.327242 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.327180    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-r2rmr\""
Jul 24 14:33:14.328076 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.328058    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 14:33:14.390022 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.389986    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:14.456898 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.456858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-accelerators-collector-config\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.457117 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.456907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-textfile\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.457117 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.456950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-tls\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.457117 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.456983    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f652c78e-d83f-40d7-9211-579f2e44c46b-sys\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.457117 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.457018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-wtmp\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.457117 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.457040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f652c78e-d83f-40d7-9211-579f2e44c46b-metrics-client-ca\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.457117 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.457070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drqw2\" (UniqueName: \"kubernetes.io/projected/f652c78e-d83f-40d7-9211-579f2e44c46b-kube-api-access-drqw2\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.457117 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.457117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f652c78e-d83f-40d7-9211-579f2e44c46b-root\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.457452 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.457141    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.492814 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.492773    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:14.558213 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f652c78e-d83f-40d7-9211-579f2e44c46b-sys\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558213 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-wtmp\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f652c78e-d83f-40d7-9211-579f2e44c46b-metrics-client-ca\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drqw2\" (UniqueName: \"kubernetes.io/projected/f652c78e-d83f-40d7-9211-579f2e44c46b-kube-api-access-drqw2\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f652c78e-d83f-40d7-9211-579f2e44c46b-root\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558317    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f652c78e-d83f-40d7-9211-579f2e44c46b-sys\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-accelerators-collector-config\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-textfile\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558390    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f652c78e-d83f-40d7-9211-579f2e44c46b-root\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558404    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-wtmp\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558462 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-tls\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.558943 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.558763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-textfile\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.559221 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.559197    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f652c78e-d83f-40d7-9211-579f2e44c46b-metrics-client-ca\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.559463 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.559439    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-accelerators-collector-config\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.561334 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.561313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-tls\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.561816 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.561790    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f652c78e-d83f-40d7-9211-579f2e44c46b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.567512 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.567489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drqw2\" (UniqueName: \"kubernetes.io/projected/f652c78e-d83f-40d7-9211-579f2e44c46b-kube-api-access-drqw2\") pod \"node-exporter-xbksj\" (UID: \"f652c78e-d83f-40d7-9211-579f2e44c46b\") " pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.589310 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.589232    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:14.633937 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.633908    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xbksj"
Jul 24 14:33:14.691117 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.690978    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:14.942315 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:14.942276    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:15.076069 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:33:15.076039    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf652c78e_d83f_40d7_9211_579f2e44c46b.slice/crio-ed7816f5d506b3fce6412d0cb24ca4cf4fa5f7547b6ca6c7717ac606b403b277 WatchSource:0}: Error finding container ed7816f5d506b3fce6412d0cb24ca4cf4fa5f7547b6ca6c7717ac606b403b277: Status 404 returned error can't find the container with id ed7816f5d506b3fce6412d0cb24ca4cf4fa5f7547b6ca6c7717ac606b403b277
Jul 24 14:33:15.262919 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.262832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:15.341846 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.341811    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 14:33:15.344335 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.344310    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.347288 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.347265    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 14:33:15.347288 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.347274    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 14:33:15.347548 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.347534    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 14:33:15.347854 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.347836    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xskq4\""
Jul 24 14:33:15.349409 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.349391    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 14:33:15.349502 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.349469    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 14:33:15.349502 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.349487    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 14:33:15.349607 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.349492    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 14:33:15.350407 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.350387    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 14:33:15.350490 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.350417    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 14:33:15.362717 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.362683    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 14:33:15.363831 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.363798    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:15.467154 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.467113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-web-config\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.467406 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.467386    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.467507 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.467494    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g75r\" (UniqueName: \"kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-kube-api-access-9g75r\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.467605 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.467591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.467700 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.467688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.467790 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.467778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.467894 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.467882    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.467993 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.467981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.468079 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.468066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.468209 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.468195    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.468681 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.468653    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.468801 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.468709    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.468801 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.468744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-config-out\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.565118 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.564929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:15.570054 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570221 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570066    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570221 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570221 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570130    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570383 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570432 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570432 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-config-out\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570509 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-web-config\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570509 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570509 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:33:15.570494    2581 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 24 14:33:15.570509 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g75r\" (UniqueName: \"kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-kube-api-access-9g75r\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570717 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570717 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:33:15.570573    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-main-tls podName:26018071-394f-4a49-ac90-79400143c3c9 nodeName:}" failed. No retries permitted until 2026-07-24 14:33:16.07054122 +0000 UTC m=+74.574259167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "26018071-394f-4a49-ac90-79400143c3c9") : secret "alertmanager-main-tls" not found
Jul 24 14:33:15.570717 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570600    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.570717 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.570626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.572013 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.571705    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.572013 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.571770    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.572013 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.571978    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.573381 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.573337    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.574247 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.574207    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.574688 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.574662    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.574798 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.574779    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.574892 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.574866    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.575220 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.575198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.575483 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.575464    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-config-out\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.575606 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.575587    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-web-config\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.580798 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.580775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g75r\" (UniqueName: \"kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-kube-api-access-9g75r\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:15.664981 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.664944    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:15.694873 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.694836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vqpl5" event={"ID":"5ed878d7-e106-4e24-a24b-46253570f910","Type":"ContainerStarted","Data":"a059197103c2b3db9662531110fc355b33343eb303f79232c7e72b5007cf9f09"}
Jul 24 14:33:15.695174 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.695127    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:33:15.696519 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.696488    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-snn2w" event={"ID":"8f983040-8d3c-4f7c-8c82-12586a4033ee","Type":"ContainerStarted","Data":"3b865fa5f8515a3572f605a3a2787e257a3e439975787c77233ffb534151a484"}
Jul 24 14:33:15.697778 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.697755    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xbksj" event={"ID":"f652c78e-d83f-40d7-9211-579f2e44c46b","Type":"ContainerStarted","Data":"ed7816f5d506b3fce6412d0cb24ca4cf4fa5f7547b6ca6c7717ac606b403b277"}
Jul 24 14:33:15.712938 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.712857    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vqpl5" podStartSLOduration=64.979643132 podStartE2EDuration="1m11.712838317s" totalImagesPullingTime="6.733195185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:15.712472938 +0000 UTC m=+74.216190899" watchObservedRunningTime="2026-07-24 14:33:15.712838317 +0000 UTC m=+74.216556278"
Jul 24 14:33:15.729483 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.729430    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-snn2w" podStartSLOduration=64.971161068 podStartE2EDuration="1m11.729408573s" totalImagesPullingTime="6.758247505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:15.728319538 +0000 UTC m=+74.232037488" watchObservedRunningTime="2026-07-24 14:33:15.729408573 +0000 UTC m=+74.233126531"
Jul 24 14:33:15.743047 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:15.743012    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:16.075068 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:16.075035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:16.077412 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:16.077388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:16.093545 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:16.093513    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:16.253789 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:16.253677    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:33:16.385566 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:16.385534    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 14:33:16.388697 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:33:16.388671    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26018071_394f_4a49_ac90_79400143c3c9.slice/crio-9ee00b7974d8a54dd716b9046152085be9a7052f39dcd46ffbcdd832801340a3 WatchSource:0}: Error finding container 9ee00b7974d8a54dd716b9046152085be9a7052f39dcd46ffbcdd832801340a3: Status 404 returned error can't find the container with id 9ee00b7974d8a54dd716b9046152085be9a7052f39dcd46ffbcdd832801340a3
Jul 24 14:33:16.702951 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:16.702919    2581 generic.go:356] "Generic (PLEG): container finished" podID="f652c78e-d83f-40d7-9211-579f2e44c46b" containerID="69b1741fa57af1a99e44c5e6c087e50d7bbb485e22af4a0b652b67b8ae769859" exitCode=0
Jul 24 14:33:16.703173 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:16.703005    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xbksj" event={"ID":"f652c78e-d83f-40d7-9211-579f2e44c46b","Type":"ContainerDied","Data":"69b1741fa57af1a99e44c5e6c087e50d7bbb485e22af4a0b652b67b8ae769859"}
Jul 24 14:33:16.704256 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:16.704228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerStarted","Data":"9ee00b7974d8a54dd716b9046152085be9a7052f39dcd46ffbcdd832801340a3"}
Jul 24 14:33:17.212252 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.212219    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:17.274102 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.274058    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-66bc45f789-8mtnb"]
Jul 24 14:33:17.276863 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.276843    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.279700 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.279673    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qtd8k\""
Jul 24 14:33:17.279700 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.279636    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 24 14:33:17.279883 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.279829    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 24 14:33:17.279883 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.279872    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 24 14:33:17.279990 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.279905    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 24 14:33:17.280223 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.280205    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cogj470i7sp63\""
Jul 24 14:33:17.280651 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.280637    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 24 14:33:17.290951 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.290924    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66bc45f789-8mtnb"]
Jul 24 14:33:17.301877 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.301851    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:17.386526 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.386486    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.386766 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.386542    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbzw8\" (UniqueName: \"kubernetes.io/projected/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-kube-api-access-pbzw8\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.386766 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.386582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.386766 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.386626    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-tls\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.386766 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.386682    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.386766 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.386722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.387015 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.386805    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-metrics-client-ca\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.387015 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.386874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-grpc-tls\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.434424 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.434389    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:17.487312 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.487286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.487456 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.487323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbzw8\" (UniqueName: \"kubernetes.io/projected/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-kube-api-access-pbzw8\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.487456 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.487341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.487456 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.487364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-tls\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.487456 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.487386    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.487456 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.487401    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.487456 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.487420    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-metrics-client-ca\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.487456 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.487447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-grpc-tls\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.488452 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.488421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-metrics-client-ca\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.490286 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.490250    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.490393 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.490304    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-tls\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.490393 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.490333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.490393 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.490355    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.490544 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.490407    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.490544 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.490489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-secret-grpc-tls\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.496060 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.496034    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbzw8\" (UniqueName: \"kubernetes.io/projected/4590de7e-2a93-4e19-98a4-eb7d3cc27c5f-kube-api-access-pbzw8\") pod \"thanos-querier-66bc45f789-8mtnb\" (UID: \"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f\") " pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.588163 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.588130    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:17.709940 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.709901    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xbksj" event={"ID":"f652c78e-d83f-40d7-9211-579f2e44c46b","Type":"ContainerStarted","Data":"e85fabda5d4a37b2f823b30dc075a3637fd24f375923986b300aeb448c4842cd"}
Jul 24 14:33:17.710149 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.709967    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xbksj" event={"ID":"f652c78e-d83f-40d7-9211-579f2e44c46b","Type":"ContainerStarted","Data":"6185cdefc17e769928ee07523ad40e75aa39267c5067d1658d6dde5f2fbee9e2"}
Jul 24 14:33:17.711398 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.711374    2581 generic.go:356] "Generic (PLEG): container finished" podID="26018071-394f-4a49-ac90-79400143c3c9" containerID="6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59" exitCode=0
Jul 24 14:33:17.711490 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.711450    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerDied","Data":"6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59"}
Jul 24 14:33:17.729642 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.729528    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66bc45f789-8mtnb"]
Jul 24 14:33:17.730375 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.730329    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xbksj" podStartSLOduration=2.864851899 podStartE2EDuration="3.730313818s" totalImagesPullingTime="800.053241ms" totalInitContainerRuntime="65.408678ms" isStatefulPod=true podCreationTimestamp="2026-07-24 14:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:17.729734077 +0000 UTC m=+76.233452073" watchObservedRunningTime="2026-07-24 14:33:17.730313818 +0000 UTC m=+76.234031786"
Jul 24 14:33:17.732020 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:17.731986    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:17.733032 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:33:17.732996    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4590de7e_2a93_4e19_98a4_eb7d3cc27c5f.slice/crio-6399eb3d5e0933b6f10de6b65e24ec2356fe1bb4d8291819a5e766ede6f55509 WatchSource:0}: Error finding container 6399eb3d5e0933b6f10de6b65e24ec2356fe1bb4d8291819a5e766ede6f55509: Status 404 returned error can't find the container with id 6399eb3d5e0933b6f10de6b65e24ec2356fe1bb4d8291819a5e766ede6f55509
Jul 24 14:33:18.716988 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:18.716944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb" event={"ID":"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f","Type":"ContainerStarted","Data":"6399eb3d5e0933b6f10de6b65e24ec2356fe1bb4d8291819a5e766ede6f55509"}
Jul 24 14:33:19.056507 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.056423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:19.068051 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.068019    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"]
Jul 24 14:33:19.071704 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.071676    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"
Jul 24 14:33:19.074509 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.074393    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 24 14:33:19.074509 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.074408    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-95lts\""
Jul 24 14:33:19.082009 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.081805    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"]
Jul 24 14:33:19.152508 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.152455    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:19.200361 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.200324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eaf81ce3-20cc-411a-b776-09d5d286e4c5-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xdz88\" (UID: \"eaf81ce3-20cc-411a-b776-09d5d286e4c5\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"
Jul 24 14:33:19.200534 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.200377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tjdc\" (UniqueName: \"kubernetes.io/projected/eaf81ce3-20cc-411a-b776-09d5d286e4c5-kube-api-access-9tjdc\") pod \"monitoring-plugin-68b7fc9574-xdz88\" (UID: \"eaf81ce3-20cc-411a-b776-09d5d286e4c5\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"
Jul 24 14:33:19.251272 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.251237    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:19.300778 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.300750    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eaf81ce3-20cc-411a-b776-09d5d286e4c5-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xdz88\" (UID: \"eaf81ce3-20cc-411a-b776-09d5d286e4c5\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"
Jul 24 14:33:19.300937 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.300798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tjdc\" (UniqueName: \"kubernetes.io/projected/eaf81ce3-20cc-411a-b776-09d5d286e4c5-kube-api-access-9tjdc\") pod \"monitoring-plugin-68b7fc9574-xdz88\" (UID: \"eaf81ce3-20cc-411a-b776-09d5d286e4c5\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"
Jul 24 14:33:19.303722 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.303696    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eaf81ce3-20cc-411a-b776-09d5d286e4c5-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xdz88\" (UID: \"eaf81ce3-20cc-411a-b776-09d5d286e4c5\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"
Jul 24 14:33:19.311145 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.311054    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tjdc\" (UniqueName: \"kubernetes.io/projected/eaf81ce3-20cc-411a-b776-09d5d286e4c5-kube-api-access-9tjdc\") pod \"monitoring-plugin-68b7fc9574-xdz88\" (UID: \"eaf81ce3-20cc-411a-b776-09d5d286e4c5\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"
Jul 24 14:33:19.385255 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.385209    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"
Jul 24 14:33:19.538622 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.538585    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-787bc58487-lhtx8"]
Jul 24 14:33:19.540833 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.540796    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.545880 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.545854    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zrd6c\""
Jul 24 14:33:19.546052 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.545895    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 24 14:33:19.546052 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.545897    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 24 14:33:19.546207 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.546118    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 24 14:33:19.546207 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.546128    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 24 14:33:19.547450 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.547421    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 24 14:33:19.553637 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.553613    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 24 14:33:19.559733 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.559710    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-787bc58487-lhtx8"]
Jul 24 14:33:19.599977 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.599869    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:19.604489 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.604456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-telemeter-client-tls\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.604648 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.604498    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d924890d-3b15-4fc5-a1b2-715405d2f28c-metrics-client-ca\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.604648 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.604566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d924890d-3b15-4fc5-a1b2-715405d2f28c-serving-certs-ca-bundle\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.604648 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.604604    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.604648 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.604646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-secret-telemeter-client\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.604851 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.604730    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-federate-client-tls\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.604851 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.604759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l65wd\" (UniqueName: \"kubernetes.io/projected/d924890d-3b15-4fc5-a1b2-715405d2f28c-kube-api-access-l65wd\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.604851 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.604831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d924890d-3b15-4fc5-a1b2-715405d2f28c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.664321 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.664292    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:19.705441 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.705400    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-federate-client-tls\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.705631 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.705446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l65wd\" (UniqueName: \"kubernetes.io/projected/d924890d-3b15-4fc5-a1b2-715405d2f28c-kube-api-access-l65wd\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.705631 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.705482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d924890d-3b15-4fc5-a1b2-715405d2f28c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.705631 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.705506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-telemeter-client-tls\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.705631 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.705529    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d924890d-3b15-4fc5-a1b2-715405d2f28c-metrics-client-ca\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.705631 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.705562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d924890d-3b15-4fc5-a1b2-715405d2f28c-serving-certs-ca-bundle\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.705631 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.705586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.705631 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.705621    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-secret-telemeter-client\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.706399 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.706368    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d924890d-3b15-4fc5-a1b2-715405d2f28c-serving-certs-ca-bundle\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.706628 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.706603    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d924890d-3b15-4fc5-a1b2-715405d2f28c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.706845 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.706800    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d924890d-3b15-4fc5-a1b2-715405d2f28c-metrics-client-ca\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.708502 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.708474    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-telemeter-client-tls\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.708502 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.708497    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-secret-telemeter-client\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.708671 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.708483    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-federate-client-tls\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.708882 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.708862    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d924890d-3b15-4fc5-a1b2-715405d2f28c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.718892 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.718734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l65wd\" (UniqueName: \"kubernetes.io/projected/d924890d-3b15-4fc5-a1b2-715405d2f28c-kube-api-access-l65wd\") pod \"telemeter-client-787bc58487-lhtx8\" (UID: \"d924890d-3b15-4fc5-a1b2-715405d2f28c\") " pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.812773 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.812734    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:19.857249 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.856606    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8"
Jul 24 14:33:19.920720 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.920587    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:19.970788 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:19.969397    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"]
Jul 24 14:33:20.014344 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.014026    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:20.045870 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.045827    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-787bc58487-lhtx8"]
Jul 24 14:33:20.050026 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:33:20.049995    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd924890d_3b15_4fc5_a1b2_715405d2f28c.slice/crio-485a7a0806c805830825d3c543cda60b9c92b131d2a5f141c951cd5edf654810 WatchSource:0}: Error finding container 485a7a0806c805830825d3c543cda60b9c92b131d2a5f141c951cd5edf654810: Status 404 returned error can't find the container with id 485a7a0806c805830825d3c543cda60b9c92b131d2a5f141c951cd5edf654810
Jul 24 14:33:20.496936 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.496835    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:20.497227 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.497208    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 14:33:20.500555 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.500505    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.503628 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.503600    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 14:33:20.503628 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.503617    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 14:33:20.504869 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.504843    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-85c8tbu5ksjfq\""
Jul 24 14:33:20.505063 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.505045    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 14:33:20.505207 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.505192    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 14:33:20.505359 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.505337    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 14:33:20.505444 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.505413    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-74hx6\""
Jul 24 14:33:20.505615 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.505595    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 14:33:20.505693 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.505683    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 14:33:20.505830 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.505809    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 14:33:20.505911 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.505885    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 24 14:33:20.506040 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.506020    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 14:33:20.506190 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.506172    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 14:33:20.506267 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.506202    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 14:33:20.506580 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.506556    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 14:33:20.508066 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.508048    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 14:33:20.509122 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.509061    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 14:33:20.522793 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.522766    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 14:33:20.614231 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614193    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614413 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614249    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614413 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-web-config\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614413 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614312    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614413 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614357    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config-out\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614413 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614428    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614624    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614849 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614680    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614849 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614731    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614849 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614761    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614849 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614809    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.614849 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64kkb\" (UniqueName: \"kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-kube-api-access-64kkb\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.615059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614866    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.615059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.615059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.615059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.614963    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.672836 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.672794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:20.715760 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.715760 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.715760 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715707    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.715760 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64kkb\" (UniqueName: \"kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-kube-api-access-64kkb\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.715760 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715790    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715815    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715938    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-web-config\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.715984    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config-out\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.716010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.716041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.716078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.716129    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.716167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.716190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.716292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.716218    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.720664 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.720629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.721114 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.720951    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.721114 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.720629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.722851 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.721409    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.722851 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.721976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.722851 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.721999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.722851 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.722177    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.722851 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.722301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.722851 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.722524    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.723271 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.723205    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.723681 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.723609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config-out\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.726251 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.726224    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.727515 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.727468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.727620 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.727566    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.728279 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.727772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.731674 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.731382    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.731674 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.731628    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88" event={"ID":"eaf81ce3-20cc-411a-b776-09d5d286e4c5","Type":"ContainerStarted","Data":"fb137b070e411f9a85809cf586cf1fe1ffa3ae357a48f330d24daa2c03a517b8"}
Jul 24 14:33:20.735323 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.735293    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-web-config\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.738388 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.738062    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.738388 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.738314    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.739008 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.738977    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64kkb\" (UniqueName: \"kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-kube-api-access-64kkb\") pod \"prometheus-k8s-0\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:20.739593 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.739563    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerStarted","Data":"9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f"}
Jul 24 14:33:20.739686 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.739644    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerStarted","Data":"ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18"}
Jul 24 14:33:20.739754 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.739690    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerStarted","Data":"086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc"}
Jul 24 14:33:20.739754 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.739730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerStarted","Data":"dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045"}
Jul 24 14:33:20.739843 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.739771    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerStarted","Data":"d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f"}
Jul 24 14:33:20.743626 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.743449    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb" event={"ID":"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f","Type":"ContainerStarted","Data":"de8fd5f5bcfcc597f7dd9e5007ddc6d5050e9bbb43cf989ea8e984cac9b4669a"}
Jul 24 14:33:20.743626 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.743532    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb" event={"ID":"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f","Type":"ContainerStarted","Data":"d2c1e346fb188a425f3bf780b5459b97dfcaeddb541a74bed38cfb2d3f599f37"}
Jul 24 14:33:20.743626 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.743573    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb" event={"ID":"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f","Type":"ContainerStarted","Data":"777a6f22a1c7479451a8e22be15956566fd14ec1a76c6e394fc5dac741142a36"}
Jul 24 14:33:20.751566 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.751057    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8" event={"ID":"d924890d-3b15-4fc5-a1b2-715405d2f28c","Type":"ContainerStarted","Data":"485a7a0806c805830825d3c543cda60b9c92b131d2a5f141c951cd5edf654810"}
Jul 24 14:33:20.775087 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.774995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:20.817512 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:20.817473    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:21.017538 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.017374    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 14:33:21.116703 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:33:21.116670    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a678c3_3aed_4d7c_b8ca_8ae24fb8fba8.slice/crio-900cad23113cc5d2e35ae1d130580e9bf1b56445e1fc540ee90085300a68a381 WatchSource:0}: Error finding container 900cad23113cc5d2e35ae1d130580e9bf1b56445e1fc540ee90085300a68a381: Status 404 returned error can't find the container with id 900cad23113cc5d2e35ae1d130580e9bf1b56445e1fc540ee90085300a68a381
Jul 24 14:33:21.763374 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.762366    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88" event={"ID":"eaf81ce3-20cc-411a-b776-09d5d286e4c5","Type":"ContainerStarted","Data":"4ffaf5e8ea71347544cfcae9e72e154b813bd9bb71234e99c868249eec437b1a"}
Jul 24 14:33:21.763374 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.763196    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"
Jul 24 14:33:21.769853 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.769822    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerStarted","Data":"6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f"}
Jul 24 14:33:21.770468 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.770447    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88"
Jul 24 14:33:21.777081 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.777045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb" event={"ID":"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f","Type":"ContainerStarted","Data":"fc1e7ae5043d4a4c39ad0bc5a752e7acf210e9a23de0a468cfbf529da0c19586"}
Jul 24 14:33:21.777235 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.777165    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb" event={"ID":"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f","Type":"ContainerStarted","Data":"54882e8d212ec1c11e763d2c0ad1d09e6bd9ac04ca1b3b3d2011874bd37c03fa"}
Jul 24 14:33:21.779778 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.779747    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerID="dbaba7b5219c4300a5ec042749b6b41a7a2d1308560ac3b6dcf9f80559337e43" exitCode=0
Jul 24 14:33:21.779925 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.779793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerDied","Data":"dbaba7b5219c4300a5ec042749b6b41a7a2d1308560ac3b6dcf9f80559337e43"}
Jul 24 14:33:21.779925 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.779856    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerStarted","Data":"900cad23113cc5d2e35ae1d130580e9bf1b56445e1fc540ee90085300a68a381"}
Jul 24 14:33:21.785308 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.785003    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xdz88" podStartSLOduration=1.259597706 podStartE2EDuration="2.784985278s" totalImagesPullingTime="1.525387572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:21.783045589 +0000 UTC m=+80.286763550" watchObservedRunningTime="2026-07-24 14:33:21.784985278 +0000 UTC m=+80.288703235"
Jul 24 14:33:21.821181 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:21.820434    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.878599661 podStartE2EDuration="6.820407012s" totalImagesPullingTime="3.924693607s" totalInitContainerRuntime="17.113744ms" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:21.819423316 +0000 UTC m=+80.323141287" watchObservedRunningTime="2026-07-24 14:33:21.820407012 +0000 UTC m=+80.324124971"
Jul 24 14:33:22.790962 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:22.790853    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb" event={"ID":"4590de7e-2a93-4e19-98a4-eb7d3cc27c5f","Type":"ContainerStarted","Data":"c1d0b1431b5335b12cb4a521de70f215f50ce724414d958e7f60271ede869cda"}
Jul 24 14:33:22.791394 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:22.791064    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:22.793417 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:22.793315    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8" event={"ID":"d924890d-3b15-4fc5-a1b2-715405d2f28c","Type":"ContainerStarted","Data":"be53bb3f00f8151df179495d068a58a405733a406cf62021f6718b4ca34b6b2f"}
Jul 24 14:33:22.793417 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:22.793371    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8" event={"ID":"d924890d-3b15-4fc5-a1b2-715405d2f28c","Type":"ContainerStarted","Data":"c89facb61df9c6a57ac18483f666b5310da525e615857352bbbc90ba7d253b4d"}
Jul 24 14:33:22.793417 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:22.793394    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8" event={"ID":"d924890d-3b15-4fc5-a1b2-715405d2f28c","Type":"ContainerStarted","Data":"2de7c58d5f81f4f48fe79a4914fcb55fac2bf4fa322df78a733656f94f49ddbc"}
Jul 24 14:33:22.814805 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:22.814740    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb" podStartSLOduration=2.284761398 podStartE2EDuration="5.814719733s" totalImagesPullingTime="3.529958335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:22.813780068 +0000 UTC m=+81.317498025" watchObservedRunningTime="2026-07-24 14:33:22.814719733 +0000 UTC m=+81.318437691"
Jul 24 14:33:22.836991 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:22.836932    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-787bc58487-lhtx8" podStartSLOduration=1.495829405 podStartE2EDuration="3.836914853s" totalImagesPullingTime="2.341085448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:22.835870309 +0000 UTC m=+81.339588268" watchObservedRunningTime="2026-07-24 14:33:22.836914853 +0000 UTC m=+81.340632809"
Jul 24 14:33:23.121897 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:23.121863    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:23.180678 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:23.180640    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:23.584059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:23.583892    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:23.685371 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:23.685338    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:25.813632 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:25.813602    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerStarted","Data":"44eeb382823510f41fc3b5fb2646670e7761fa41930db0f0a82310313c039755"}
Jul 24 14:33:25.814007 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:25.813696    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerStarted","Data":"5314809293afebe0a017595582b2cff33180e9d335daa3eb252fc3887cd2c490"}
Jul 24 14:33:25.814007 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:25.813730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerStarted","Data":"1ec2e6dbe1d5072b96c0f4ae826917c2674349f40803dc542954a6c9b7142e28"}
Jul 24 14:33:25.814007 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:25.813770    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerStarted","Data":"10381a620517799d68e5b1104b6acc7d72ad591c2a5bc8af2a10eadc5fada79b"}
Jul 24 14:33:25.814007 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:25.813806    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerStarted","Data":"9917cd1eb7be0625d05113d7243347b77ff518be77acb6fb9d06cbe4e2c2cb94"}
Jul 24 14:33:25.955514 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:25.955424    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:26.042277 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:26.042241    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:26.823105 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:26.823059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerStarted","Data":"924b2cda7260c546399f58dea7cca07be60063c888e087e1c17c208c8e13de38"}
Jul 24 14:33:26.855968 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:26.855899    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.217351579 podStartE2EDuration="6.855877519s" totalImagesPullingTime="3.612983124s" totalInitContainerRuntime="25.542816ms" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:26.854405773 +0000 UTC m=+85.358123757" watchObservedRunningTime="2026-07-24 14:33:26.855877519 +0000 UTC m=+85.359595479"
Jul 24 14:33:26.998310 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:26.998276    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:28.419840 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:28.419804    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:28.520394 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:28.520361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:28.806114 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:28.806015    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-66bc45f789-8mtnb"
Jul 24 14:33:29.753986 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:29.753939    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:29.813661 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:29.813623    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:30.634423 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:30.634390    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:30.742499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:30.742466    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:30.818253 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:30.818216    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:33:30.834504 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:30.834472    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:32.679627 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:32.679593    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:32.804608 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:32.804574    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:34.791521 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:34.791490    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:34.892751 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:34.892685    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:36.762599 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:36.762560    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:36.862914 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:36.862877    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:36.965753 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:36.965707    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:45.632164 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:45.632126    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:45.729217 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:45.729174    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:46.706838 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:46.706809    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vqpl5"
Jul 24 14:33:47.884702 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:47.884671    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:33:47.949178 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:33:47.949147    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:20.818619 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:20.818583    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:20.837575 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:20.837546    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:21.049941 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:21.049913    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:30.407474 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:30.407438    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:31.741231 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:31.741194    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:34.670180 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:34.670144    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 14:34:34.670747 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:34.670561    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="alertmanager" containerID="cri-o://d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f" gracePeriod=120
Jul 24 14:34:34.670747 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:34.670594    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy" containerID="cri-o://ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18" gracePeriod=120
Jul 24 14:34:34.670747 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:34.670569    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="config-reloader" containerID="cri-o://dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045" gracePeriod=120
Jul 24 14:34:34.670747 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:34.670566    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy-web" containerID="cri-o://086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc" gracePeriod=120
Jul 24 14:34:34.670962 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:34.670779    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy-metric" containerID="cri-o://9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f" gracePeriod=120
Jul 24 14:34:34.670962 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:34.670818    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="prom-label-proxy" containerID="cri-o://6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f" gracePeriod=120
Jul 24 14:34:35.088447 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.088359    2581 generic.go:356] "Generic (PLEG): container finished" podID="26018071-394f-4a49-ac90-79400143c3c9" containerID="6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f" exitCode=0
Jul 24 14:34:35.088447 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.088384    2581 generic.go:356] "Generic (PLEG): container finished" podID="26018071-394f-4a49-ac90-79400143c3c9" containerID="ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18" exitCode=0
Jul 24 14:34:35.088447 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.088390    2581 generic.go:356] "Generic (PLEG): container finished" podID="26018071-394f-4a49-ac90-79400143c3c9" containerID="dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045" exitCode=0
Jul 24 14:34:35.088447 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.088396    2581 generic.go:356] "Generic (PLEG): container finished" podID="26018071-394f-4a49-ac90-79400143c3c9" containerID="d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f" exitCode=0
Jul 24 14:34:35.088447 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.088427    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerDied","Data":"6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f"}
Jul 24 14:34:35.088741 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.088505    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerDied","Data":"ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18"}
Jul 24 14:34:35.088741 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.088549    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerDied","Data":"dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045"}
Jul 24 14:34:35.088741 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.088585    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerDied","Data":"d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f"}
Jul 24 14:34:35.907023 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.906998    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:35.948558 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.948528    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-config-out\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.948798 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.948777    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-main-db\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.948920 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.948902    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.949114 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.949076    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.949290 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.949212    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:34:35.949428 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.949278    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-cluster-tls-config\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.949635 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.949463    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-metrics-client-ca\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.949635 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.949526    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-tls-assets\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.949635 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.949555    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-main-tls\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.949635 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.949603    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-trusted-ca-bundle\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.949635 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.949632    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9g75r\" (UniqueName: \"kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-kube-api-access-9g75r\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.949904 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.949700    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-config-volume\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.949904 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.949771    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-web-config\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.949904 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.949810    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"26018071-394f-4a49-ac90-79400143c3c9\" (UID: \"26018071-394f-4a49-ac90-79400143c3c9\") "
Jul 24 14:34:35.950059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.950038    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:34:35.950222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.950199    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:34:35.950895 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.950568    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-main-db\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:35.950895 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.950591    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-metrics-client-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:35.950895 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.950604    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26018071-394f-4a49-ac90-79400143c3c9-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:35.951973 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.951929    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:35.953395 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.953364    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:35.953801 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.953766    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-config-out" (OuterVolumeSpecName: "config-out") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:34:35.954106 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.954021    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:35.954498 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.954473    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:35.955573 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.955515    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-config-volume" (OuterVolumeSpecName: "config-volume") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:35.956635 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.956610    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-kube-api-access-9g75r" (OuterVolumeSpecName: "kube-api-access-9g75r") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "kube-api-access-9g75r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:34:35.959318 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.959268    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:34:35.961549 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.961528    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:35.967453 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:35.967429    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-web-config" (OuterVolumeSpecName: "web-config") pod "26018071-394f-4a49-ac90-79400143c3c9" (UID: "26018071-394f-4a49-ac90-79400143c3c9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:36.051226 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.051145    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26018071-394f-4a49-ac90-79400143c3c9-config-out\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:36.051226 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.051170    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:36.051226 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.051181    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:36.051226 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.051191    2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-cluster-tls-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:36.051226 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.051200    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-tls-assets\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:36.051226 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.051209    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-main-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:36.051226 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.051217    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g75r\" (UniqueName: \"kubernetes.io/projected/26018071-394f-4a49-ac90-79400143c3c9-kube-api-access-9g75r\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:36.051226 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.051225    2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-config-volume\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:36.051226 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.051233    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-web-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:36.051588 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.051241    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/26018071-394f-4a49-ac90-79400143c3c9-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:36.095242 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.095204    2581 generic.go:356] "Generic (PLEG): container finished" podID="26018071-394f-4a49-ac90-79400143c3c9" containerID="9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f" exitCode=0
Jul 24 14:34:36.095242 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.095232    2581 generic.go:356] "Generic (PLEG): container finished" podID="26018071-394f-4a49-ac90-79400143c3c9" containerID="086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc" exitCode=0
Jul 24 14:34:36.095467 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.095265    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerDied","Data":"9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f"}
Jul 24 14:34:36.095467 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.095286    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.095467 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.095368    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerDied","Data":"086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc"}
Jul 24 14:34:36.095467 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.095409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"26018071-394f-4a49-ac90-79400143c3c9","Type":"ContainerDied","Data":"9ee00b7974d8a54dd716b9046152085be9a7052f39dcd46ffbcdd832801340a3"}
Jul 24 14:34:36.095467 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.095438    2581 scope.go:117] "RemoveContainer" containerID="6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f"
Jul 24 14:34:36.103557 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.103541    2581 scope.go:117] "RemoveContainer" containerID="9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f"
Jul 24 14:34:36.110575 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.110560    2581 scope.go:117] "RemoveContainer" containerID="ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18"
Jul 24 14:34:36.116925 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.116909    2581 scope.go:117] "RemoveContainer" containerID="086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc"
Jul 24 14:34:36.122815 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.122788    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 14:34:36.124603 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.124579    2581 scope.go:117] "RemoveContainer" containerID="dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045"
Jul 24 14:34:36.128879 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.128852    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 14:34:36.131762 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.131744    2581 scope.go:117] "RemoveContainer" containerID="d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f"
Jul 24 14:34:36.138267 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.138246    2581 scope.go:117] "RemoveContainer" containerID="6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59"
Jul 24 14:34:36.144698 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.144664    2581 scope.go:117] "RemoveContainer" containerID="6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f"
Jul 24 14:34:36.144940 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:34:36.144920    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f\": container with ID starting with 6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f not found: ID does not exist" containerID="6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f"
Jul 24 14:34:36.145004 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.144949    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f"} err="failed to get container status \"6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f\": rpc error: code = NotFound desc = could not find container \"6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f\": container with ID starting with 6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f not found: ID does not exist"
Jul 24 14:34:36.145004 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.144978    2581 scope.go:117] "RemoveContainer" containerID="9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f"
Jul 24 14:34:36.145262 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:34:36.145248    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f\": container with ID starting with 9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f not found: ID does not exist" containerID="9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f"
Jul 24 14:34:36.145315 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.145267    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f"} err="failed to get container status \"9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f\": rpc error: code = NotFound desc = could not find container \"9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f\": container with ID starting with 9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f not found: ID does not exist"
Jul 24 14:34:36.145315 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.145286    2581 scope.go:117] "RemoveContainer" containerID="ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18"
Jul 24 14:34:36.145529 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:34:36.145515    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18\": container with ID starting with ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18 not found: ID does not exist" containerID="ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18"
Jul 24 14:34:36.145573 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.145530    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18"} err="failed to get container status \"ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18\": rpc error: code = NotFound desc = could not find container \"ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18\": container with ID starting with ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18 not found: ID does not exist"
Jul 24 14:34:36.145573 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.145543    2581 scope.go:117] "RemoveContainer" containerID="086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc"
Jul 24 14:34:36.145769 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:34:36.145749    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc\": container with ID starting with 086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc not found: ID does not exist" containerID="086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc"
Jul 24 14:34:36.145833 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.145778    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc"} err="failed to get container status \"086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc\": rpc error: code = NotFound desc = could not find container \"086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc\": container with ID starting with 086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc not found: ID does not exist"
Jul 24 14:34:36.145833 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.145802    2581 scope.go:117] "RemoveContainer" containerID="dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045"
Jul 24 14:34:36.146068 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:34:36.146053    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045\": container with ID starting with dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045 not found: ID does not exist" containerID="dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045"
Jul 24 14:34:36.146142 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.146070    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045"} err="failed to get container status \"dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045\": rpc error: code = NotFound desc = could not find container \"dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045\": container with ID starting with dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045 not found: ID does not exist"
Jul 24 14:34:36.146142 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.146083    2581 scope.go:117] "RemoveContainer" containerID="d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f"
Jul 24 14:34:36.146365 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:34:36.146336    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f\": container with ID starting with d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f not found: ID does not exist" containerID="d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f"
Jul 24 14:34:36.146411 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.146363    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f"} err="failed to get container status \"d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f\": rpc error: code = NotFound desc = could not find container \"d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f\": container with ID starting with d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f not found: ID does not exist"
Jul 24 14:34:36.146411 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.146380    2581 scope.go:117] "RemoveContainer" containerID="6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59"
Jul 24 14:34:36.146652 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:34:36.146633    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59\": container with ID starting with 6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59 not found: ID does not exist" containerID="6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59"
Jul 24 14:34:36.146710 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.146656    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59"} err="failed to get container status \"6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59\": rpc error: code = NotFound desc = could not find container \"6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59\": container with ID starting with 6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59 not found: ID does not exist"
Jul 24 14:34:36.146710 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.146681    2581 scope.go:117] "RemoveContainer" containerID="6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f"
Jul 24 14:34:36.146919 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.146901    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f"} err="failed to get container status \"6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f\": rpc error: code = NotFound desc = could not find container \"6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f\": container with ID starting with 6c9d68c46ab22ff8a25b2f33695777ec13aad18a0dcf0175c0a7d6fa141dae6f not found: ID does not exist"
Jul 24 14:34:36.146971 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.146920    2581 scope.go:117] "RemoveContainer" containerID="9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f"
Jul 24 14:34:36.147169 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.147118    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f"} err="failed to get container status \"9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f\": rpc error: code = NotFound desc = could not find container \"9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f\": container with ID starting with 9f547eeda5c83b99842a446c6561fbfd2b38fcafaa8973d119433ecfa20b8f5f not found: ID does not exist"
Jul 24 14:34:36.147169 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.147133    2581 scope.go:117] "RemoveContainer" containerID="ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18"
Jul 24 14:34:36.147352 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.147337    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18"} err="failed to get container status \"ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18\": rpc error: code = NotFound desc = could not find container \"ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18\": container with ID starting with ee88b54576ae8ef39742bb4714e6cd703ae7204aad9bbaac9ea5691403f23e18 not found: ID does not exist"
Jul 24 14:34:36.147392 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.147352    2581 scope.go:117] "RemoveContainer" containerID="086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc"
Jul 24 14:34:36.147561 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.147545    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc"} err="failed to get container status \"086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc\": rpc error: code = NotFound desc = could not find container \"086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc\": container with ID starting with 086a830ad2c96960cc69602cbde412740d6c31b07d71197d1b38a053eece8bfc not found: ID does not exist"
Jul 24 14:34:36.147595 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.147562    2581 scope.go:117] "RemoveContainer" containerID="dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045"
Jul 24 14:34:36.147796 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.147779    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045"} err="failed to get container status \"dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045\": rpc error: code = NotFound desc = could not find container \"dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045\": container with ID starting with dac124531960aa58e9b416e3997e0a1b63e782d903f2cd84c203eb773ef9f045 not found: ID does not exist"
Jul 24 14:34:36.147842 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.147797    2581 scope.go:117] "RemoveContainer" containerID="d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f"
Jul 24 14:34:36.148006 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.147990    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f"} err="failed to get container status \"d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f\": rpc error: code = NotFound desc = could not find container \"d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f\": container with ID starting with d20b6c3233298ab9ccfc34c0967e944f21138a377dd503be354eab51b4a8b99f not found: ID does not exist"
Jul 24 14:34:36.148049 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.148006    2581 scope.go:117] "RemoveContainer" containerID="6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59"
Jul 24 14:34:36.148230 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.148211    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59"} err="failed to get container status \"6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59\": rpc error: code = NotFound desc = could not find container \"6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59\": container with ID starting with 6ca7124e87b98238ff7ad3cad5bf6dd4243a21cc7ab8bd5e5d591e359aa83e59 not found: ID does not exist"
Jul 24 14:34:36.162154 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162135    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 14:34:36.162416 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162402    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="prom-label-proxy"
Jul 24 14:34:36.162499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162418    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="prom-label-proxy"
Jul 24 14:34:36.162499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162441    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="alertmanager"
Jul 24 14:34:36.162499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162449    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="alertmanager"
Jul 24 14:34:36.162499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162457    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy-metric"
Jul 24 14:34:36.162499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162463    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy-metric"
Jul 24 14:34:36.162499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162473    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="init-config-reloader"
Jul 24 14:34:36.162499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162478    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="init-config-reloader"
Jul 24 14:34:36.162499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162485    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy-web"
Jul 24 14:34:36.162499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162490    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy-web"
Jul 24 14:34:36.162499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162498    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="config-reloader"
Jul 24 14:34:36.162499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162504    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="config-reloader"
Jul 24 14:34:36.163014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162512    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy"
Jul 24 14:34:36.163014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162520    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy"
Jul 24 14:34:36.163014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162566    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="config-reloader"
Jul 24 14:34:36.163014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162574    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="prom-label-proxy"
Jul 24 14:34:36.163014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162582    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="alertmanager"
Jul 24 14:34:36.163014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162592    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy-web"
Jul 24 14:34:36.163014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162600    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy"
Jul 24 14:34:36.163014 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.162608    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="26018071-394f-4a49-ac90-79400143c3c9" containerName="kube-rbac-proxy-metric"
Jul 24 14:34:36.167135 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.167116    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.169805 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.169783    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 14:34:36.170546 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.170530    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 14:34:36.170546 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.170545    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 14:34:36.170734 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.170589    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xskq4\""
Jul 24 14:34:36.170734 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.170615    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 14:34:36.170882 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.170867    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 14:34:36.170945 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.170870    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 14:34:36.170945 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.170872    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 14:34:36.171351 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.171335    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 14:34:36.180303 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.180277    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 14:34:36.183837 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.183816    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 14:34:36.252830 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.252793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.252830 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.252833    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.253058 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.252860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51d07f2b-dec7-485f-a725-93d754508b1c-config-out\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.253058 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.252883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51d07f2b-dec7-485f-a725-93d754508b1c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.253058 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.252921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq865\" (UniqueName: \"kubernetes.io/projected/51d07f2b-dec7-485f-a725-93d754508b1c-kube-api-access-rq865\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.253058 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.252998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.253058 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.253020    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.253058 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.253040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51d07f2b-dec7-485f-a725-93d754508b1c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.253058 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.253059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-config-volume\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.253385 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.253076    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51d07f2b-dec7-485f-a725-93d754508b1c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.253385 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.253145    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.253385 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.253213    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51d07f2b-dec7-485f-a725-93d754508b1c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.253385 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.253245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-web-config\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.354692 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.354597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51d07f2b-dec7-485f-a725-93d754508b1c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.354692 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.354638    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-web-config\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.354692 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.354659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.354931 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.354782    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.354931 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.354820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51d07f2b-dec7-485f-a725-93d754508b1c-config-out\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.355038 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.354938    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51d07f2b-dec7-485f-a725-93d754508b1c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.355038 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.354971    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq865\" (UniqueName: \"kubernetes.io/projected/51d07f2b-dec7-485f-a725-93d754508b1c-kube-api-access-rq865\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.355186 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.355117    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.355186 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.355154    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.355291 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.355185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51d07f2b-dec7-485f-a725-93d754508b1c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.355291 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.355219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-config-volume\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.355291 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.355242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51d07f2b-dec7-485f-a725-93d754508b1c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.355291 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.355266    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.355476 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.355320    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/51d07f2b-dec7-485f-a725-93d754508b1c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.357655 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.357624    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51d07f2b-dec7-485f-a725-93d754508b1c-config-out\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.357782 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.357730    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51d07f2b-dec7-485f-a725-93d754508b1c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.357782 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.357752    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.357782 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.357753    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-web-config\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.357950 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.357912    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.357950 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.357924    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.358392 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.358363    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.358500 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.358371    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51d07f2b-dec7-485f-a725-93d754508b1c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.358500 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.358468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51d07f2b-dec7-485f-a725-93d754508b1c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.358595 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.358578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-config-volume\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.359625 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.359608    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/51d07f2b-dec7-485f-a725-93d754508b1c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.364210 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.364192    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq865\" (UniqueName: \"kubernetes.io/projected/51d07f2b-dec7-485f-a725-93d754508b1c-kube-api-access-rq865\") pod \"alertmanager-main-0\" (UID: \"51d07f2b-dec7-485f-a725-93d754508b1c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.476684 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.476653    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 14:34:36.616412 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:36.616381    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 14:34:36.619349 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:34:36.619318    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d07f2b_dec7_485f_a725_93d754508b1c.slice/crio-e76a98770b308e85e886c2ec373914c23b3e6eb85efab0d684c78124dc4fa093 WatchSource:0}: Error finding container e76a98770b308e85e886c2ec373914c23b3e6eb85efab0d684c78124dc4fa093: Status 404 returned error can't find the container with id e76a98770b308e85e886c2ec373914c23b3e6eb85efab0d684c78124dc4fa093
Jul 24 14:34:37.101636 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:37.101550    2581 generic.go:356] "Generic (PLEG): container finished" podID="51d07f2b-dec7-485f-a725-93d754508b1c" containerID="9c61b47eb2f604578ccaccb5962d1db0f0db03be4bf9e63226d8d979474f68c4" exitCode=0
Jul 24 14:34:37.101636 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:37.101585    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51d07f2b-dec7-485f-a725-93d754508b1c","Type":"ContainerDied","Data":"9c61b47eb2f604578ccaccb5962d1db0f0db03be4bf9e63226d8d979474f68c4"}
Jul 24 14:34:37.102012 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:37.101653    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51d07f2b-dec7-485f-a725-93d754508b1c","Type":"ContainerStarted","Data":"e76a98770b308e85e886c2ec373914c23b3e6eb85efab0d684c78124dc4fa093"}
Jul 24 14:34:37.333139 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:37.333086    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26018071-394f-4a49-ac90-79400143c3c9" path="/var/lib/kubelet/pods/26018071-394f-4a49-ac90-79400143c3c9/volumes"
Jul 24 14:34:38.108677 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.108645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51d07f2b-dec7-485f-a725-93d754508b1c","Type":"ContainerStarted","Data":"19517ac34f994b112d9db2b4ada0bdc4ed8387af635f1b65c391366b5e37679f"}
Jul 24 14:34:38.109010 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.108706    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51d07f2b-dec7-485f-a725-93d754508b1c","Type":"ContainerStarted","Data":"79187f65b20103ae060ce64082695e1db124bebbde1c57b8a4060b68b230b324"}
Jul 24 14:34:38.109010 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.108733    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51d07f2b-dec7-485f-a725-93d754508b1c","Type":"ContainerStarted","Data":"e1d2a5cbc533c53f70669f394a05c123277b313bb3fd5fcd53ad19333cad25d4"}
Jul 24 14:34:38.109010 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.108758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51d07f2b-dec7-485f-a725-93d754508b1c","Type":"ContainerStarted","Data":"5766e35b717ca0501d895b8603bcbdbc79e465599677ca18b676f67785afa5ea"}
Jul 24 14:34:38.109010 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.108785    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51d07f2b-dec7-485f-a725-93d754508b1c","Type":"ContainerStarted","Data":"b49760bacb73b0c904526c84f68b26e4a9266ce960ba982c0fa4552a5bc7694e"}
Jul 24 14:34:38.109010 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.108808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"51d07f2b-dec7-485f-a725-93d754508b1c","Type":"ContainerStarted","Data":"2a735546ce0379f958d8e2c91f21a4eac9ffdfec2efc057b2ad3101d0a2f7e8d"}
Jul 24 14:34:38.139213 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.139162    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.120188949 podStartE2EDuration="2.139147371s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.958422ms" isStatefulPod=false podCreationTimestamp="2026-07-24 14:34:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:34:38.136550744 +0000 UTC m=+156.640268714" watchObservedRunningTime="2026-07-24 14:34:38.139147371 +0000 UTC m=+156.642865328"
Jul 24 14:34:38.948352 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.948315    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 14:34:38.949059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.948681    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="prometheus" containerID="cri-o://9917cd1eb7be0625d05113d7243347b77ff518be77acb6fb9d06cbe4e2c2cb94" gracePeriod=600
Jul 24 14:34:38.949059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.948706    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy" containerID="cri-o://44eeb382823510f41fc3b5fb2646670e7761fa41930db0f0a82310313c039755" gracePeriod=600
Jul 24 14:34:38.949059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.948861    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy-web" containerID="cri-o://5314809293afebe0a017595582b2cff33180e9d335daa3eb252fc3887cd2c490" gracePeriod=600
Jul 24 14:34:38.949059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.948895    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="config-reloader" containerID="cri-o://10381a620517799d68e5b1104b6acc7d72ad591c2a5bc8af2a10eadc5fada79b" gracePeriod=600
Jul 24 14:34:38.949059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.948863    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://924b2cda7260c546399f58dea7cca07be60063c888e087e1c17c208c8e13de38" gracePeriod=600
Jul 24 14:34:38.949059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:38.948973    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="thanos-sidecar" containerID="cri-o://1ec2e6dbe1d5072b96c0f4ae826917c2674349f40803dc542954a6c9b7142e28" gracePeriod=600
Jul 24 14:34:39.117030 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.116999    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerID="924b2cda7260c546399f58dea7cca07be60063c888e087e1c17c208c8e13de38" exitCode=0
Jul 24 14:34:39.117030 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.117024    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerID="44eeb382823510f41fc3b5fb2646670e7761fa41930db0f0a82310313c039755" exitCode=0
Jul 24 14:34:39.117030 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.117030    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerID="5314809293afebe0a017595582b2cff33180e9d335daa3eb252fc3887cd2c490" exitCode=0
Jul 24 14:34:39.117030 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.117036    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerID="1ec2e6dbe1d5072b96c0f4ae826917c2674349f40803dc542954a6c9b7142e28" exitCode=0
Jul 24 14:34:39.117030 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.117041    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerID="10381a620517799d68e5b1104b6acc7d72ad591c2a5bc8af2a10eadc5fada79b" exitCode=0
Jul 24 14:34:39.117030 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.117046    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerID="9917cd1eb7be0625d05113d7243347b77ff518be77acb6fb9d06cbe4e2c2cb94" exitCode=0
Jul 24 14:34:39.117558 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.117075    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerDied","Data":"924b2cda7260c546399f58dea7cca07be60063c888e087e1c17c208c8e13de38"}
Jul 24 14:34:39.117558 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.117161    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerDied","Data":"44eeb382823510f41fc3b5fb2646670e7761fa41930db0f0a82310313c039755"}
Jul 24 14:34:39.117558 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.117189    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerDied","Data":"5314809293afebe0a017595582b2cff33180e9d335daa3eb252fc3887cd2c490"}
Jul 24 14:34:39.117558 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.117213    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerDied","Data":"1ec2e6dbe1d5072b96c0f4ae826917c2674349f40803dc542954a6c9b7142e28"}
Jul 24 14:34:39.117558 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.117239    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerDied","Data":"10381a620517799d68e5b1104b6acc7d72ad591c2a5bc8af2a10eadc5fada79b"}
Jul 24 14:34:39.117558 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.117261    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerDied","Data":"9917cd1eb7be0625d05113d7243347b77ff518be77acb6fb9d06cbe4e2c2cb94"}
Jul 24 14:34:39.216148 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.216123    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:39.278062 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278025    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-2\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278225 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278073    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-kube-rbac-proxy\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278273 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278242    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-kubelet-serving-ca-bundle\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278317 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278293    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278369 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278328    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config-out\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278369 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278355    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-tls\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278467 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278379    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:34:39.278467 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278388    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-thanos-prometheus-http-client-file\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278467 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278414    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-db\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278467 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278442    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64kkb\" (UniqueName: \"kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-kube-api-access-64kkb\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278662 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278475    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-serving-certs-ca-bundle\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278662 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278509    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-web-config\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278662 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278535    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278662 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278574    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-metrics-client-ca\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278662 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278607    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-1\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.278662 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278609    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:34:39.278662 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278654    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-metrics-client-certs\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.279022 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278706    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-trusted-ca-bundle\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.279022 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278761    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-tls-assets\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.279022 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278786    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.279022 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278811    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-grpc-tls\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.279022 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.278867    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-0\") pod \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\" (UID: \"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8\") "
Jul 24 14:34:39.279281 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.279138    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.279281 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.279139    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:34:39.279281 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.279158    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.281116 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.281061    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:34:39.285175 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.282186    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:34:39.285175 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.282225    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:34:39.285175 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.282601    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:34:39.285175 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.284570    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:39.285175 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.284785    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:39.285175 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.284812    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:34:39.286779 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.285766    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:39.287048 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.286919    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:39.287048 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.287007    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:39.287048 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.287020    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config" (OuterVolumeSpecName: "config") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:39.287367 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.287062    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config-out" (OuterVolumeSpecName: "config-out") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:34:39.287367 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.287110    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:39.287367 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.287194    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:34:39.287367 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.287258    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-kube-api-access-64kkb" (OuterVolumeSpecName: "kube-api-access-64kkb") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "kube-api-access-64kkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:34:39.293628 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.293574    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:39.299525 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.299502    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-web-config" (OuterVolumeSpecName: "web-config") pod "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" (UID: "f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:34:39.380584 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380548    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-tls-assets\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380584 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380587    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380603    2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-grpc-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380623    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380636    2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-kube-rbac-proxy\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380649    2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380662    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-config-out\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380676    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380689    2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380703    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-db\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380716    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64kkb\" (UniqueName: \"kubernetes.io/projected/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-kube-api-access-64kkb\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380733    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380747    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-web-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380761    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380775    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-configmap-metrics-client-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380788    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380801    2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-secret-metrics-client-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:39.380822 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:39.380811    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:34:40.123731 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.123688    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8","Type":"ContainerDied","Data":"900cad23113cc5d2e35ae1d130580e9bf1b56445e1fc540ee90085300a68a381"}
Jul 24 14:34:40.124083 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.123768    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.124083 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.123786    2581 scope.go:117] "RemoveContainer" containerID="924b2cda7260c546399f58dea7cca07be60063c888e087e1c17c208c8e13de38"
Jul 24 14:34:40.131461 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.131444    2581 scope.go:117] "RemoveContainer" containerID="44eeb382823510f41fc3b5fb2646670e7761fa41930db0f0a82310313c039755"
Jul 24 14:34:40.138186 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.138167    2581 scope.go:117] "RemoveContainer" containerID="5314809293afebe0a017595582b2cff33180e9d335daa3eb252fc3887cd2c490"
Jul 24 14:34:40.144384 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.144365    2581 scope.go:117] "RemoveContainer" containerID="1ec2e6dbe1d5072b96c0f4ae826917c2674349f40803dc542954a6c9b7142e28"
Jul 24 14:34:40.147659 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.147639    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 14:34:40.151763 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.151743    2581 scope.go:117] "RemoveContainer" containerID="10381a620517799d68e5b1104b6acc7d72ad591c2a5bc8af2a10eadc5fada79b"
Jul 24 14:34:40.152917 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.152896    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 14:34:40.158124 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.158103    2581 scope.go:117] "RemoveContainer" containerID="9917cd1eb7be0625d05113d7243347b77ff518be77acb6fb9d06cbe4e2c2cb94"
Jul 24 14:34:40.164719 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.164700    2581 scope.go:117] "RemoveContainer" containerID="dbaba7b5219c4300a5ec042749b6b41a7a2d1308560ac3b6dcf9f80559337e43"
Jul 24 14:34:40.205595 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.205565    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 14:34:40.205901 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.205886    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="init-config-reloader"
Jul 24 14:34:40.205968 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.205903    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="init-config-reloader"
Jul 24 14:34:40.205968 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.205921    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy"
Jul 24 14:34:40.205968 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.205931    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy"
Jul 24 14:34:40.205968 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.205946    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="prometheus"
Jul 24 14:34:40.205968 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.205955    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="prometheus"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.205969    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="config-reloader"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.205977    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="config-reloader"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.205987    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="thanos-sidecar"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.205995    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="thanos-sidecar"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.206010    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy-thanos"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.206020    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy-thanos"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.206033    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy-web"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.206041    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy-web"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.206125    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="thanos-sidecar"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.206140    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="prometheus"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.206152    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.206162    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy-thanos"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.206171    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="config-reloader"
Jul 24 14:34:40.206222 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.206180    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" containerName="kube-rbac-proxy-web"
Jul 24 14:34:40.210156 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.210137    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.212905 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.212884    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-74hx6\""
Jul 24 14:34:40.213345 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.213302    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 14:34:40.213661 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.213648    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 14:34:40.213855 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.213843    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 14:34:40.214133 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.214112    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 14:34:40.214210 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.214186    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 14:34:40.214557 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.214543    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 14:34:40.221108 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.221075    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 14:34:40.222137 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.222121    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-85c8tbu5ksjfq\""
Jul 24 14:34:40.222257 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.222241    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 14:34:40.222392 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.222372    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 24 14:34:40.222392 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.222387    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 14:34:40.222479 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.222400    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 14:34:40.222479 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.222387    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 14:34:40.222479 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.222390    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 14:34:40.230708 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.230691    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 14:34:40.243953 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.243929    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 14:34:40.247608 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.247592    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 14:34:40.287412 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287412 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287543 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287543 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b862ffde-981d-4e89-8708-440f247e3a9b-config-out\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287543 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287509    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287710 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287545    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-config\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287710 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287710 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287649    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287710 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rszjl\" (UniqueName: \"kubernetes.io/projected/b862ffde-981d-4e89-8708-440f247e3a9b-kube-api-access-rszjl\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287853 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b862ffde-981d-4e89-8708-440f247e3a9b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287853 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287742    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287853 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287853 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287816    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.287853 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287842    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.288013 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287862    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-web-config\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.288013 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.288013 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287920    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.288013 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.288013 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287965    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.288013 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.287987    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.388830 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.388796    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.388982 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.388838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.388982 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.388867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.388982 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.388894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.388982 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.388915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-web-config\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.388982 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.388946    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389269 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389269 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389059    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389269 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389269 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389157    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389269 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389269 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389210    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389269 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389240    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389616 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389271    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b862ffde-981d-4e89-8708-440f247e3a9b-config-out\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389616 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389616 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-config\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389616 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389349    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389616 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389616 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389438    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rszjl\" (UniqueName: \"kubernetes.io/projected/b862ffde-981d-4e89-8708-440f247e3a9b-kube-api-access-rszjl\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389616 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b862ffde-981d-4e89-8708-440f247e3a9b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389894 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389645    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389894 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389777    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.389993 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.389909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.390584 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.390501    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.391954 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.390963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.392050 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.392000    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.392379 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.392228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.392886 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.392633    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b862ffde-981d-4e89-8708-440f247e3a9b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.392886 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.392710    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.392886 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.392811    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.392886 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.392848    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.393155 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.392992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.393155 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.393049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-web-config\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.394064 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.394040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b862ffde-981d-4e89-8708-440f247e3a9b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.394204 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.394148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b862ffde-981d-4e89-8708-440f247e3a9b-config-out\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.394726 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.394702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.395288 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.395260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.395440 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.395417    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-config\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.395549 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.395530    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b862ffde-981d-4e89-8708-440f247e3a9b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.405162 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.405143    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rszjl\" (UniqueName: \"kubernetes.io/projected/b862ffde-981d-4e89-8708-440f247e3a9b-kube-api-access-rszjl\") pod \"prometheus-k8s-0\" (UID: \"b862ffde-981d-4e89-8708-440f247e3a9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.519021 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.518928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:40.660501 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:40.660399    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 14:34:40.662460 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:34:40.662438    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb862ffde_981d_4e89_8708_440f247e3a9b.slice/crio-0c21b01538a929de29992da338310013e0512334fbc5ba0e3b5b0ed2d964fbc4 WatchSource:0}: Error finding container 0c21b01538a929de29992da338310013e0512334fbc5ba0e3b5b0ed2d964fbc4: Status 404 returned error can't find the container with id 0c21b01538a929de29992da338310013e0512334fbc5ba0e3b5b0ed2d964fbc4
Jul 24 14:34:41.135143 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:41.135087    2581 generic.go:356] "Generic (PLEG): container finished" podID="b862ffde-981d-4e89-8708-440f247e3a9b" containerID="4094d0c2643ee0e54536fe4535b5ac0e7f59582f7dc2cb7327063a7b748012f5" exitCode=0
Jul 24 14:34:41.135569 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:41.135161    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b862ffde-981d-4e89-8708-440f247e3a9b","Type":"ContainerDied","Data":"4094d0c2643ee0e54536fe4535b5ac0e7f59582f7dc2cb7327063a7b748012f5"}
Jul 24 14:34:41.135569 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:41.135252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b862ffde-981d-4e89-8708-440f247e3a9b","Type":"ContainerStarted","Data":"0c21b01538a929de29992da338310013e0512334fbc5ba0e3b5b0ed2d964fbc4"}
Jul 24 14:34:41.335482 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:41.335453    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8" path="/var/lib/kubelet/pods/f7a678c3-3aed-4d7c-b8ca-8ae24fb8fba8/volumes"
Jul 24 14:34:42.143024 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:42.142993    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b862ffde-981d-4e89-8708-440f247e3a9b","Type":"ContainerStarted","Data":"013903b01f1a9e3e99f0a05ac9bfd1e8d188697bebc1b26f250c5d76991d0be0"}
Jul 24 14:34:42.143461 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:42.143054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b862ffde-981d-4e89-8708-440f247e3a9b","Type":"ContainerStarted","Data":"46704aeadcf001943b3922e6a40aab8889fdb9b697593df1e861ff6f6d45f807"}
Jul 24 14:34:42.143461 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:42.143117    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b862ffde-981d-4e89-8708-440f247e3a9b","Type":"ContainerStarted","Data":"70f7a827970ab8cae48472ae97b56ebc1d98bf24b4e0a563f67a95be1cf5ae23"}
Jul 24 14:34:42.143461 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:42.143148    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b862ffde-981d-4e89-8708-440f247e3a9b","Type":"ContainerStarted","Data":"f2c73e6e53fb9fb5692f89dd3e05f7ba13b8e48d7506766113b50892a0a3c746"}
Jul 24 14:34:42.143461 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:42.143173    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b862ffde-981d-4e89-8708-440f247e3a9b","Type":"ContainerStarted","Data":"4fd06ce3c70f611d1d9ddb98a86ff753503cd214751dfa07c15974db4712903b"}
Jul 24 14:34:42.143461 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:42.143198    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b862ffde-981d-4e89-8708-440f247e3a9b","Type":"ContainerStarted","Data":"9bc4ee7bee7d5331ae3fd87f77ca809cea2d0e0e04b59ec7cdfc5da44c8d0732"}
Jul 24 14:34:42.180643 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:42.180557    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.162014782 podStartE2EDuration="2.180534342s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.51956ms" isStatefulPod=false podCreationTimestamp="2026-07-24 14:34:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:34:42.177701537 +0000 UTC m=+160.681419494" watchObservedRunningTime="2026-07-24 14:34:42.180534342 +0000 UTC m=+160.684252300"
Jul 24 14:34:42.364846 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:42.364807    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:42.433846 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:42.433760    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:45.519338 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:45.519306    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:34:45.647449 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:45.647420    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:34:45.748408 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:34:45.748368    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:01.688805 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:01.688715    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:12.653444 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:12.653409    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:12.788369 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:12.788334    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:13.004011 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:13.003920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:13.157547 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:13.157516    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:13.269275 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:13.269189    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:13.367907 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:13.367869    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:30.957355 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:30.957307    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:31.053041 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:31.053014    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:40.520010 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:40.519979    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:35:40.536287 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:40.536262    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:35:41.382778 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:41.382750    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 14:35:43.099616 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:43.099577    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:43.170863 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:43.170830    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:53.009628 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:53.009596    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:53.096844 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:53.096805    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:53.755231 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:53.755195    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:56.614150 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:56.614108    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:56.712783 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:56.712752    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:58.010206 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:58.010170    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:58.105150 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:58.105114    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:58.554983 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:58.554951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:35:58.662135 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:35:58.662102    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:00.266474 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:00.266444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:00.439477 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:00.439444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:03.370176 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:03.370138    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:03.443254 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:03.443216    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:08.210533 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:08.210501    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:09.018510 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.018471    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rmwkp"]
Jul 24 14:36:09.020477 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.020462    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rmwkp"
Jul 24 14:36:09.023064 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.023043    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 14:36:09.023299 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.023279    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-pmc99\""
Jul 24 14:36:09.023384 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.023367    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 14:36:09.032015 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.031995    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rmwkp"]
Jul 24 14:36:09.146230 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.146192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjh7z\" (UniqueName: \"kubernetes.io/projected/8a196956-b479-47b3-aeea-8ae7d440ceef-kube-api-access-bjh7z\") pod \"cert-manager-55b4d954d5-rmwkp\" (UID: \"8a196956-b479-47b3-aeea-8ae7d440ceef\") " pod="cert-manager/cert-manager-55b4d954d5-rmwkp"
Jul 24 14:36:09.146401 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.146299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a196956-b479-47b3-aeea-8ae7d440ceef-bound-sa-token\") pod \"cert-manager-55b4d954d5-rmwkp\" (UID: \"8a196956-b479-47b3-aeea-8ae7d440ceef\") " pod="cert-manager/cert-manager-55b4d954d5-rmwkp"
Jul 24 14:36:09.247174 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.247142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjh7z\" (UniqueName: \"kubernetes.io/projected/8a196956-b479-47b3-aeea-8ae7d440ceef-kube-api-access-bjh7z\") pod \"cert-manager-55b4d954d5-rmwkp\" (UID: \"8a196956-b479-47b3-aeea-8ae7d440ceef\") " pod="cert-manager/cert-manager-55b4d954d5-rmwkp"
Jul 24 14:36:09.247513 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.247207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a196956-b479-47b3-aeea-8ae7d440ceef-bound-sa-token\") pod \"cert-manager-55b4d954d5-rmwkp\" (UID: \"8a196956-b479-47b3-aeea-8ae7d440ceef\") " pod="cert-manager/cert-manager-55b4d954d5-rmwkp"
Jul 24 14:36:09.255970 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.255947    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a196956-b479-47b3-aeea-8ae7d440ceef-bound-sa-token\") pod \"cert-manager-55b4d954d5-rmwkp\" (UID: \"8a196956-b479-47b3-aeea-8ae7d440ceef\") " pod="cert-manager/cert-manager-55b4d954d5-rmwkp"
Jul 24 14:36:09.261186 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.256446    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjh7z\" (UniqueName: \"kubernetes.io/projected/8a196956-b479-47b3-aeea-8ae7d440ceef-kube-api-access-bjh7z\") pod \"cert-manager-55b4d954d5-rmwkp\" (UID: \"8a196956-b479-47b3-aeea-8ae7d440ceef\") " pod="cert-manager/cert-manager-55b4d954d5-rmwkp"
Jul 24 14:36:09.341688 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.341617    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rmwkp"
Jul 24 14:36:09.474179 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:09.474149    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rmwkp"]
Jul 24 14:36:09.476964 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:36:09.476943    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a196956_b479_47b3_aeea_8ae7d440ceef.slice/crio-9aec6d66619d3df8d78b242c15e8433ffe60134e7ae86379e34ba7bb975c7df0 WatchSource:0}: Error finding container 9aec6d66619d3df8d78b242c15e8433ffe60134e7ae86379e34ba7bb975c7df0: Status 404 returned error can't find the container with id 9aec6d66619d3df8d78b242c15e8433ffe60134e7ae86379e34ba7bb975c7df0
Jul 24 14:36:10.479328 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:10.479286    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rmwkp" event={"ID":"8a196956-b479-47b3-aeea-8ae7d440ceef","Type":"ContainerStarted","Data":"9aec6d66619d3df8d78b242c15e8433ffe60134e7ae86379e34ba7bb975c7df0"}
Jul 24 14:36:13.493621 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:13.493584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rmwkp" event={"ID":"8a196956-b479-47b3-aeea-8ae7d440ceef","Type":"ContainerStarted","Data":"5289634c4547cf477e422015998a86ee40bd6442279126edf6d7d52c4e31a640"}
Jul 24 14:36:13.511774 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:13.511727    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rmwkp" podStartSLOduration=1.23806116 podStartE2EDuration="4.511712338s" totalImagesPullingTime="3.273651178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:36:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:36:13.510379391 +0000 UTC m=+252.014097348" watchObservedRunningTime="2026-07-24 14:36:13.511712338 +0000 UTC m=+252.015430295"
Jul 24 14:36:15.198394 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:15.198345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:15.299349 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:15.299296    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:20.161958 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:20.161925    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:20.228497 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:20.228463    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:20.904571 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:20.904528    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:21.017934 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:21.017900    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:22.797801 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:22.797764    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:33.176363 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.176270    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:33.236973 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.236936    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"]
Jul 24 14:36:33.241686 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.241661    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.245117 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.245079    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:36:33.246072 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.246052    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bkrnw\""
Jul 24 14:36:33.246325 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.246305    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 14:36:33.246325 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.246320    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 14:36:33.246463 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.246377    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 14:36:33.246463 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.246424    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 14:36:33.261087 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.261068    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"]
Jul 24 14:36:33.299400 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.299372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:33.339211 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.339180    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a3a460e-7b62-4aee-b635-bfe181c84281-manager-config\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.339211 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.339210    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a3a460e-7b62-4aee-b635-bfe181c84281-metrics-cert\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.339412 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.339238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a3a460e-7b62-4aee-b635-bfe181c84281-cert\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.339412 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.339339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p85v\" (UniqueName: \"kubernetes.io/projected/8a3a460e-7b62-4aee-b635-bfe181c84281-kube-api-access-4p85v\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.439834 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.439747    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a3a460e-7b62-4aee-b635-bfe181c84281-manager-config\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.439834 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.439784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a3a460e-7b62-4aee-b635-bfe181c84281-metrics-cert\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.439834 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.439804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a3a460e-7b62-4aee-b635-bfe181c84281-cert\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.440143 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.439852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p85v\" (UniqueName: \"kubernetes.io/projected/8a3a460e-7b62-4aee-b635-bfe181c84281-kube-api-access-4p85v\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.440457 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.440433    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a3a460e-7b62-4aee-b635-bfe181c84281-manager-config\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.442438 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.442414    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a3a460e-7b62-4aee-b635-bfe181c84281-cert\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.442527 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.442462    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a3a460e-7b62-4aee-b635-bfe181c84281-metrics-cert\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.449754 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.449734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p85v\" (UniqueName: \"kubernetes.io/projected/8a3a460e-7b62-4aee-b635-bfe181c84281-kube-api-access-4p85v\") pod \"lws-controller-manager-55664b4b95-v9r5m\" (UID: \"8a3a460e-7b62-4aee-b635-bfe181c84281\") " pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.550880 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.550832    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:33.693204 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:33.693126    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"]
Jul 24 14:36:33.696124 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:36:33.696075    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a3a460e_7b62_4aee_b635_bfe181c84281.slice/crio-5306e4e9c43684dadb40f675106d454af6ac3b959c2f6c78f81a3e696e0f1db2 WatchSource:0}: Error finding container 5306e4e9c43684dadb40f675106d454af6ac3b959c2f6c78f81a3e696e0f1db2: Status 404 returned error can't find the container with id 5306e4e9c43684dadb40f675106d454af6ac3b959c2f6c78f81a3e696e0f1db2
Jul 24 14:36:34.579590 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:34.579546    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m" event={"ID":"8a3a460e-7b62-4aee-b635-bfe181c84281","Type":"ContainerStarted","Data":"5306e4e9c43684dadb40f675106d454af6ac3b959c2f6c78f81a3e696e0f1db2"}
Jul 24 14:36:36.591561 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:36.591527    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m" event={"ID":"8a3a460e-7b62-4aee-b635-bfe181c84281","Type":"ContainerStarted","Data":"eb11d9fb3d0f1361d89957d9d67d9838e386b5e2bb0a163539043f809eeb116f"}
Jul 24 14:36:36.591955 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:36.591601    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:36.616352 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:36.616301    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m" podStartSLOduration=1.255086396 podStartE2EDuration="3.616287312s" totalImagesPullingTime="2.361200916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:36:36.615344532 +0000 UTC m=+275.119062500" watchObservedRunningTime="2026-07-24 14:36:36.616287312 +0000 UTC m=+275.120005270"
Jul 24 14:36:37.440594 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:37.440559    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:37.541921 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:37.541887    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:38.076819 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.076782    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:38.350796 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.350679    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:38.448218 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.448157    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:38.493433 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.493407    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"]
Jul 24 14:36:38.495683 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.495666    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.499341 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.499322    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 14:36:38.499664 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.499649    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p884f\""
Jul 24 14:36:38.500050 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.500032    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 14:36:38.500131 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.500115    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 14:36:38.501353 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.501332    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 14:36:38.519921 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.519887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"]
Jul 24 14:36:38.587108 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.587058    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b7741a8-9b14-4135-95fe-bf60afa9bc74-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.587108 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.587110    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7741a8-9b14-4135-95fe-bf60afa9bc74-tmp\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.587328 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.587132    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b7741a8-9b14-4135-95fe-bf60afa9bc74-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.587328 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.587207    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ckxv\" (UniqueName: \"kubernetes.io/projected/4b7741a8-9b14-4135-95fe-bf60afa9bc74-kube-api-access-7ckxv\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.587328 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.587271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b7741a8-9b14-4135-95fe-bf60afa9bc74-manifests\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.673818 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.673786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:38.687830 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.687805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ckxv\" (UniqueName: \"kubernetes.io/projected/4b7741a8-9b14-4135-95fe-bf60afa9bc74-kube-api-access-7ckxv\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.687920 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.687853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b7741a8-9b14-4135-95fe-bf60afa9bc74-manifests\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.687920 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.687895    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b7741a8-9b14-4135-95fe-bf60afa9bc74-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.687920 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.687915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7741a8-9b14-4135-95fe-bf60afa9bc74-tmp\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.688039 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.687936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b7741a8-9b14-4135-95fe-bf60afa9bc74-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.688393 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.688373    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b7741a8-9b14-4135-95fe-bf60afa9bc74-manifests\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.690136 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.690112    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7741a8-9b14-4135-95fe-bf60afa9bc74-tmp\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.690396 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.690378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b7741a8-9b14-4135-95fe-bf60afa9bc74-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.690466 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.690446    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b7741a8-9b14-4135-95fe-bf60afa9bc74-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.701440 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.701422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ckxv\" (UniqueName: \"kubernetes.io/projected/4b7741a8-9b14-4135-95fe-bf60afa9bc74-kube-api-access-7ckxv\") pod \"opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp\" (UID: \"4b7741a8-9b14-4135-95fe-bf60afa9bc74\") " pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.783540 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.783500    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:38.805595 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.805573    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:38.907932 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.907889    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:38.946311 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:38.945727    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"]
Jul 24 14:36:38.951278 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:36:38.951251    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b7741a8_9b14_4135_95fe_bf60afa9bc74.slice/crio-f955127e4bb1875652a17bd0cc81112fe4339a561eb5dfd5b14e6606ffbb5c36 WatchSource:0}: Error finding container f955127e4bb1875652a17bd0cc81112fe4339a561eb5dfd5b14e6606ffbb5c36: Status 404 returned error can't find the container with id f955127e4bb1875652a17bd0cc81112fe4339a561eb5dfd5b14e6606ffbb5c36
Jul 24 14:36:39.007881 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:39.007838    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:39.109168 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:39.109137    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:39.209302 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:39.209221    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:39.607166 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:39.607056    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp" event={"ID":"4b7741a8-9b14-4135-95fe-bf60afa9bc74","Type":"ContainerStarted","Data":"f955127e4bb1875652a17bd0cc81112fe4339a561eb5dfd5b14e6606ffbb5c36"}
Jul 24 14:36:42.622690 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:42.622652    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp" event={"ID":"4b7741a8-9b14-4135-95fe-bf60afa9bc74","Type":"ContainerStarted","Data":"43fa3f106c11bccb28b2128dcc38c36deb057456e6259feb5d968c5946b9774d"}
Jul 24 14:36:43.628125 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:43.628074    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b7741a8-9b14-4135-95fe-bf60afa9bc74" containerID="43fa3f106c11bccb28b2128dcc38c36deb057456e6259feb5d968c5946b9774d" exitCode=0
Jul 24 14:36:43.628549 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:43.628162    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp" event={"ID":"4b7741a8-9b14-4135-95fe-bf60afa9bc74","Type":"ContainerDied","Data":"43fa3f106c11bccb28b2128dcc38c36deb057456e6259feb5d968c5946b9774d"}
Jul 24 14:36:44.633954 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:44.633923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp" event={"ID":"4b7741a8-9b14-4135-95fe-bf60afa9bc74","Type":"ContainerStarted","Data":"3a64c9da2a9f739195f5f0307bcde838d10623d3c879236e016a479360f886ee"}
Jul 24 14:36:44.634366 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:44.634052    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:44.660108 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:44.660035    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp" podStartSLOduration=1.833415284 podStartE2EDuration="6.660019736s" totalImagesPullingTime="3.75171555s" totalInitContainerRuntime="1.074888902s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:36:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:36:44.658027123 +0000 UTC m=+283.161745105" watchObservedRunningTime="2026-07-24 14:36:44.660019736 +0000 UTC m=+283.163737737"
Jul 24 14:36:46.827641 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:46.827596    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:46.896551 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:46.896515    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:47.598051 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:47.598003    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-55664b4b95-v9r5m"
Jul 24 14:36:55.640569 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:55.640537    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp"
Jul 24 14:36:56.103235 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:56.103143    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:57.493198 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:57.493167    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:36:57.628083 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:36:57.628048    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:00.862870 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:00.862829    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:00.899688 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:00.899649    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"]
Jul 24 14:37:00.908155 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:00.908132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"
Jul 24 14:37:00.913193 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:00.913165    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 14:37:00.913329 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:00.913178    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2z9rw\""
Jul 24 14:37:00.913329 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:00.913256    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 14:37:00.913871 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:00.913845    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 14:37:00.914141 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:00.914126    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 14:37:00.921388 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:00.921367    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"]
Jul 24 14:37:00.996449 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:00.996415    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:01.079458 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.079424    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70f0840-aeb0-4b5e-b183-df0046491fbf-tmp\") pod \"kube-auth-proxy-55ffdfd896-w6h5l\" (UID: \"a70f0840-aeb0-4b5e-b183-df0046491fbf\") " pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"
Jul 24 14:37:01.079590 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.079467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a70f0840-aeb0-4b5e-b183-df0046491fbf-tls-certs\") pod \"kube-auth-proxy-55ffdfd896-w6h5l\" (UID: \"a70f0840-aeb0-4b5e-b183-df0046491fbf\") " pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"
Jul 24 14:37:01.079590 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.079530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j47zq\" (UniqueName: \"kubernetes.io/projected/a70f0840-aeb0-4b5e-b183-df0046491fbf-kube-api-access-j47zq\") pod \"kube-auth-proxy-55ffdfd896-w6h5l\" (UID: \"a70f0840-aeb0-4b5e-b183-df0046491fbf\") " pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"
Jul 24 14:37:01.180360 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.180281    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j47zq\" (UniqueName: \"kubernetes.io/projected/a70f0840-aeb0-4b5e-b183-df0046491fbf-kube-api-access-j47zq\") pod \"kube-auth-proxy-55ffdfd896-w6h5l\" (UID: \"a70f0840-aeb0-4b5e-b183-df0046491fbf\") " pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"
Jul 24 14:37:01.180360 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.180359    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70f0840-aeb0-4b5e-b183-df0046491fbf-tmp\") pod \"kube-auth-proxy-55ffdfd896-w6h5l\" (UID: \"a70f0840-aeb0-4b5e-b183-df0046491fbf\") " pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"
Jul 24 14:37:01.180536 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.180389    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a70f0840-aeb0-4b5e-b183-df0046491fbf-tls-certs\") pod \"kube-auth-proxy-55ffdfd896-w6h5l\" (UID: \"a70f0840-aeb0-4b5e-b183-df0046491fbf\") " pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"
Jul 24 14:37:01.182572 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.182543    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a70f0840-aeb0-4b5e-b183-df0046491fbf-tmp\") pod \"kube-auth-proxy-55ffdfd896-w6h5l\" (UID: \"a70f0840-aeb0-4b5e-b183-df0046491fbf\") " pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"
Jul 24 14:37:01.182762 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.182745    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a70f0840-aeb0-4b5e-b183-df0046491fbf-tls-certs\") pod \"kube-auth-proxy-55ffdfd896-w6h5l\" (UID: \"a70f0840-aeb0-4b5e-b183-df0046491fbf\") " pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"
Jul 24 14:37:01.199709 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.199680    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j47zq\" (UniqueName: \"kubernetes.io/projected/a70f0840-aeb0-4b5e-b183-df0046491fbf-kube-api-access-j47zq\") pod \"kube-auth-proxy-55ffdfd896-w6h5l\" (UID: \"a70f0840-aeb0-4b5e-b183-df0046491fbf\") " pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"
Jul 24 14:37:01.217552 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.217530    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"
Jul 24 14:37:01.224463 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.224437    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:01.313232 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.313181    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:01.347197 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.347168    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l"]
Jul 24 14:37:01.350730 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:37:01.350707    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda70f0840_aeb0_4b5e_b183_df0046491fbf.slice/crio-45117e0672714a4de66d26d45c12cc4c2b40b1df1648464d34977df775c141e7 WatchSource:0}: Error finding container 45117e0672714a4de66d26d45c12cc4c2b40b1df1648464d34977df775c141e7: Status 404 returned error can't find the container with id 45117e0672714a4de66d26d45c12cc4c2b40b1df1648464d34977df775c141e7
Jul 24 14:37:01.411578 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.411542    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:01.701900 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.701864    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l" event={"ID":"a70f0840-aeb0-4b5e-b183-df0046491fbf","Type":"ContainerStarted","Data":"45117e0672714a4de66d26d45c12cc4c2b40b1df1648464d34977df775c141e7"}
Jul 24 14:37:01.957936 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:01.957850    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:02.214929 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:02.214846    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:02.696820 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:02.696762    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:02.796905 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:02.796868    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:03.215948 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:03.215922    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 14:37:03.901368 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:03.901349    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 14:37:04.018213 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:04.018180    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:04.717902 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:04.717870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l" event={"ID":"a70f0840-aeb0-4b5e-b183-df0046491fbf","Type":"ContainerStarted","Data":"7cbbbdf1270c93ed777878c46096ff494578c86f4aac124cb5bd9deea4db9b6a"}
Jul 24 14:37:04.739249 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:04.739191    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-55ffdfd896-w6h5l" podStartSLOduration=2.193098062 podStartE2EDuration="4.739173645s" totalImagesPullingTime="2.546075583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:37:04.739010837 +0000 UTC m=+303.242728832" watchObservedRunningTime="2026-07-24 14:37:04.739173645 +0000 UTC m=+303.242891604"
Jul 24 14:37:07.940896 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:07.940855    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:08.061783 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:08.061744    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:08.976935 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:08.976904    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:09.190104 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:09.190068    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:09.289474 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:09.289383    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:11.620349 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.620318    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:11.689070 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.689037    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"]
Jul 24 14:37:11.691321 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.691306    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"
Jul 24 14:37:11.694047 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.694027    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 24 14:37:11.694200 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.694183    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-zx4gz\""
Jul 24 14:37:11.708984 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.708961    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"]
Jul 24 14:37:11.711313 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.711293    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"
Jul 24 14:37:11.716404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.716386    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 24 14:37:11.716630 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.716618    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-bc847\""
Jul 24 14:37:11.728587 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.728567    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"]
Jul 24 14:37:11.742001 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.741980    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"]
Jul 24 14:37:11.756475 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.756446    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:11.767672 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.767652    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2ea64ef-1529-4ef7-a910-17a76d0bb6b3-cert\") pod \"kserve-controller-manager-d6f4b65b8-wxdd4\" (UID: \"e2ea64ef-1529-4ef7-a910-17a76d0bb6b3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"
Jul 24 14:37:11.767774 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.767677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a89d208d-ce0f-4a37-9472-a194a89a111d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hk8t8\" (UID: \"a89d208d-ce0f-4a37-9472-a194a89a111d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"
Jul 24 14:37:11.767774 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.767742    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9ssp\" (UniqueName: \"kubernetes.io/projected/e2ea64ef-1529-4ef7-a910-17a76d0bb6b3-kube-api-access-z9ssp\") pod \"kserve-controller-manager-d6f4b65b8-wxdd4\" (UID: \"e2ea64ef-1529-4ef7-a910-17a76d0bb6b3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"
Jul 24 14:37:11.767774 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.767770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdl5v\" (UniqueName: \"kubernetes.io/projected/a89d208d-ce0f-4a37-9472-a194a89a111d-kube-api-access-pdl5v\") pod \"llmisvc-controller-manager-777c6bd7d5-hk8t8\" (UID: \"a89d208d-ce0f-4a37-9472-a194a89a111d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"
Jul 24 14:37:11.868904 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.868866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9ssp\" (UniqueName: \"kubernetes.io/projected/e2ea64ef-1529-4ef7-a910-17a76d0bb6b3-kube-api-access-z9ssp\") pod \"kserve-controller-manager-d6f4b65b8-wxdd4\" (UID: \"e2ea64ef-1529-4ef7-a910-17a76d0bb6b3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"
Jul 24 14:37:11.869071 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.868916    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdl5v\" (UniqueName: \"kubernetes.io/projected/a89d208d-ce0f-4a37-9472-a194a89a111d-kube-api-access-pdl5v\") pod \"llmisvc-controller-manager-777c6bd7d5-hk8t8\" (UID: \"a89d208d-ce0f-4a37-9472-a194a89a111d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"
Jul 24 14:37:11.869071 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.868947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2ea64ef-1529-4ef7-a910-17a76d0bb6b3-cert\") pod \"kserve-controller-manager-d6f4b65b8-wxdd4\" (UID: \"e2ea64ef-1529-4ef7-a910-17a76d0bb6b3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"
Jul 24 14:37:11.869071 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.868965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a89d208d-ce0f-4a37-9472-a194a89a111d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hk8t8\" (UID: \"a89d208d-ce0f-4a37-9472-a194a89a111d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"
Jul 24 14:37:11.869212 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:37:11.869083    2581 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 24 14:37:11.869212 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:37:11.869112    2581 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 24 14:37:11.869212 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:37:11.869182    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2ea64ef-1529-4ef7-a910-17a76d0bb6b3-cert podName:e2ea64ef-1529-4ef7-a910-17a76d0bb6b3 nodeName:}" failed. No retries permitted until 2026-07-24 14:37:12.369161255 +0000 UTC m=+310.872879206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e2ea64ef-1529-4ef7-a910-17a76d0bb6b3-cert") pod "kserve-controller-manager-d6f4b65b8-wxdd4" (UID: "e2ea64ef-1529-4ef7-a910-17a76d0bb6b3") : secret "kserve-webhook-server-cert" not found
Jul 24 14:37:11.869212 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:37:11.869202    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a89d208d-ce0f-4a37-9472-a194a89a111d-cert podName:a89d208d-ce0f-4a37-9472-a194a89a111d nodeName:}" failed. No retries permitted until 2026-07-24 14:37:12.369193176 +0000 UTC m=+310.872911116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a89d208d-ce0f-4a37-9472-a194a89a111d-cert") pod "llmisvc-controller-manager-777c6bd7d5-hk8t8" (UID: "a89d208d-ce0f-4a37-9472-a194a89a111d") : secret "llmisvc-webhook-server-cert" not found
Jul 24 14:37:11.883192 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.883130    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9ssp\" (UniqueName: \"kubernetes.io/projected/e2ea64ef-1529-4ef7-a910-17a76d0bb6b3-kube-api-access-z9ssp\") pod \"kserve-controller-manager-d6f4b65b8-wxdd4\" (UID: \"e2ea64ef-1529-4ef7-a910-17a76d0bb6b3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"
Jul 24 14:37:11.888428 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.888407    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdl5v\" (UniqueName: \"kubernetes.io/projected/a89d208d-ce0f-4a37-9472-a194a89a111d-kube-api-access-pdl5v\") pod \"llmisvc-controller-manager-777c6bd7d5-hk8t8\" (UID: \"a89d208d-ce0f-4a37-9472-a194a89a111d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"
Jul 24 14:37:11.988310 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:11.988279    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:12.088499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.088464    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:12.373638 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.373605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2ea64ef-1529-4ef7-a910-17a76d0bb6b3-cert\") pod \"kserve-controller-manager-d6f4b65b8-wxdd4\" (UID: \"e2ea64ef-1529-4ef7-a910-17a76d0bb6b3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"
Jul 24 14:37:12.373793 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.373642    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a89d208d-ce0f-4a37-9472-a194a89a111d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hk8t8\" (UID: \"a89d208d-ce0f-4a37-9472-a194a89a111d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"
Jul 24 14:37:12.376028 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.375994    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2ea64ef-1529-4ef7-a910-17a76d0bb6b3-cert\") pod \"kserve-controller-manager-d6f4b65b8-wxdd4\" (UID: \"e2ea64ef-1529-4ef7-a910-17a76d0bb6b3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"
Jul 24 14:37:12.376171 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.375999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a89d208d-ce0f-4a37-9472-a194a89a111d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hk8t8\" (UID: \"a89d208d-ce0f-4a37-9472-a194a89a111d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"
Jul 24 14:37:12.430725 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.430687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:12.535585 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.535539    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:12.601327 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.601294    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"
Jul 24 14:37:12.620457 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.620434    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"
Jul 24 14:37:12.759026 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.759000    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"]
Jul 24 14:37:12.760475 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:37:12.760439    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ea64ef_1529_4ef7_a910_17a76d0bb6b3.slice/crio-0f611b6ffad0f333159a26520baf5e04c31cbc4e2ddc0b20b9e10c985c774f1a WatchSource:0}: Error finding container 0f611b6ffad0f333159a26520baf5e04c31cbc4e2ddc0b20b9e10c985c774f1a: Status 404 returned error can't find the container with id 0f611b6ffad0f333159a26520baf5e04c31cbc4e2ddc0b20b9e10c985c774f1a
Jul 24 14:37:12.761713 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.761694    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 14:37:12.780960 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.780940    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"]
Jul 24 14:37:12.782419 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:37:12.782386    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda89d208d_ce0f_4a37_9472_a194a89a111d.slice/crio-e5becd7b0cc4d2da05fbddbf40fe29b80503aa17b7452e989f3658328d83fafc WatchSource:0}: Error finding container e5becd7b0cc4d2da05fbddbf40fe29b80503aa17b7452e989f3658328d83fafc: Status 404 returned error can't find the container with id e5becd7b0cc4d2da05fbddbf40fe29b80503aa17b7452e989f3658328d83fafc
Jul 24 14:37:12.967517 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:12.967437    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:13.186720 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:13.186682    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:13.757520 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:13.757484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4" event={"ID":"e2ea64ef-1529-4ef7-a910-17a76d0bb6b3","Type":"ContainerStarted","Data":"0f611b6ffad0f333159a26520baf5e04c31cbc4e2ddc0b20b9e10c985c774f1a"}
Jul 24 14:37:13.758899 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:13.758870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8" event={"ID":"a89d208d-ce0f-4a37-9472-a194a89a111d","Type":"ContainerStarted","Data":"e5becd7b0cc4d2da05fbddbf40fe29b80503aa17b7452e989f3658328d83fafc"}
Jul 24 14:37:15.776209 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:15.776170    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8" event={"ID":"a89d208d-ce0f-4a37-9472-a194a89a111d","Type":"ContainerStarted","Data":"aea74b24167c11cf6720d6cc6db7475f1ced653cdbb125f996907774091f26f2"}
Jul 24 14:37:15.776727 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:15.776278    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"
Jul 24 14:37:15.777897 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:15.777876    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4" event={"ID":"e2ea64ef-1529-4ef7-a910-17a76d0bb6b3","Type":"ContainerStarted","Data":"1f497f9f660da570c58ce2df08b5a1dc7728e600eec0d0624a966f19b97b217b"}
Jul 24 14:37:15.778023 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:15.778007    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"
Jul 24 14:37:15.849317 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:15.849261    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8" podStartSLOduration=2.019975853 podStartE2EDuration="4.849243015s" totalImagesPullingTime="2.829267162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:37:15.804867403 +0000 UTC m=+314.308585361" watchObservedRunningTime="2026-07-24 14:37:15.849243015 +0000 UTC m=+314.352960981"
Jul 24 14:37:15.850027 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:15.849990    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4" podStartSLOduration=2.648498949 podStartE2EDuration="4.849977868s" totalImagesPullingTime="2.201478919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:37:15.844756606 +0000 UTC m=+314.348474564" watchObservedRunningTime="2026-07-24 14:37:15.849977868 +0000 UTC m=+314.353695826"
Jul 24 14:37:46.785866 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:46.785839    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hk8t8"
Jul 24 14:37:46.788911 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:46.788893    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxdd4"
Jul 24 14:37:58.949499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:58.949413    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:59.159200 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:59.159166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:37:59.265747 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:37:59.265652    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:00.021633 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:00.021600    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:00.124110 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:00.124074    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:08.025041 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:08.025007    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:08.087278 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:08.087245    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:28.363869 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:28.363801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:31.969808 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:31.969762    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:32.096333 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:32.096300    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:36.979665 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:36.979629    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:37.224109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:37.224067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:39.538144 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:39.538106    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:39.678497 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:39.678454    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:40.412335 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:40.412305    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:41.839551 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:41.839522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:44.216641 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:44.216602    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:44.383201 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:44.383166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:45.360296 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:45.360258    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:45.613081 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:45.613005    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:47.364302 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:47.364267    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:47.465576 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:47.465545    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:53.421911 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:53.421877    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:53.519718 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:53.519687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:38:57.700740 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:38:57.700696    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:00.000080 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:00.000045    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:00.098859 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:00.098828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:01.226515 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:01.226477    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:01.327500 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:01.327465    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:02.585189 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:02.585152    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:02.690357 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:02.690321    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:05.596483 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:05.596454    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:05.684282 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:05.684239    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:12.667981 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:12.667942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:12.778346 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:12.778318    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:19.211023 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:19.210980    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:19.323018 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:19.322981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:35.982533 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:35.982454    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:38.552926 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.552893    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:38.779387 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.779355    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:39:38.782522 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.782506    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"
Jul 24 14:39:38.784850 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.784827    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 14:39:38.785005 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.784857    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dgd8w\""
Jul 24 14:39:38.785005 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.784879    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 14:39:38.785005 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.784920    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 14:39:38.795796 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.795777    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:39:38.806412 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.806360    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:39:38.853152 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.853125    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrttv\" (UniqueName: \"kubernetes.io/projected/745928f7-2edc-42d3-8be0-d3cbe0ba5988-kube-api-access-vrttv\") pod \"limitador-limitador-54d6d988d5-5jwnw\" (UID: \"745928f7-2edc-42d3-8be0-d3cbe0ba5988\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"
Jul 24 14:39:38.853289 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.853193    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/745928f7-2edc-42d3-8be0-d3cbe0ba5988-config-file\") pod \"limitador-limitador-54d6d988d5-5jwnw\" (UID: \"745928f7-2edc-42d3-8be0-d3cbe0ba5988\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"
Jul 24 14:39:38.954298 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.954266    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/745928f7-2edc-42d3-8be0-d3cbe0ba5988-config-file\") pod \"limitador-limitador-54d6d988d5-5jwnw\" (UID: \"745928f7-2edc-42d3-8be0-d3cbe0ba5988\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"
Jul 24 14:39:38.954463 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.954360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrttv\" (UniqueName: \"kubernetes.io/projected/745928f7-2edc-42d3-8be0-d3cbe0ba5988-kube-api-access-vrttv\") pod \"limitador-limitador-54d6d988d5-5jwnw\" (UID: \"745928f7-2edc-42d3-8be0-d3cbe0ba5988\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"
Jul 24 14:39:38.954944 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.954926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/745928f7-2edc-42d3-8be0-d3cbe0ba5988-config-file\") pod \"limitador-limitador-54d6d988d5-5jwnw\" (UID: \"745928f7-2edc-42d3-8be0-d3cbe0ba5988\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"
Jul 24 14:39:38.962291 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:38.962263    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrttv\" (UniqueName: \"kubernetes.io/projected/745928f7-2edc-42d3-8be0-d3cbe0ba5988-kube-api-access-vrttv\") pod \"limitador-limitador-54d6d988d5-5jwnw\" (UID: \"745928f7-2edc-42d3-8be0-d3cbe0ba5988\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"
Jul 24 14:39:39.093206 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.093129    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"
Jul 24 14:39:39.204671 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.204637    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-kq5jm"]
Jul 24 14:39:39.208590 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.208565    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kq5jm"
Jul 24 14:39:39.211118 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.211082    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xwk6h\""
Jul 24 14:39:39.216716 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.216690    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kq5jm"]
Jul 24 14:39:39.237035 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.237017    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:39:39.248132 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.248085    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:39.257967 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.257946    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkpvj\" (UniqueName: \"kubernetes.io/projected/c1d48542-340c-417a-b7c7-c0a7c016582b-kube-api-access-fkpvj\") pod \"authorino-58cd955675-kq5jm\" (UID: \"c1d48542-340c-417a-b7c7-c0a7c016582b\") " pod="kuadrant-system/authorino-58cd955675-kq5jm"
Jul 24 14:39:39.307736 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.307700    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:39.359282 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.359254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkpvj\" (UniqueName: \"kubernetes.io/projected/c1d48542-340c-417a-b7c7-c0a7c016582b-kube-api-access-fkpvj\") pod \"authorino-58cd955675-kq5jm\" (UID: \"c1d48542-340c-417a-b7c7-c0a7c016582b\") " pod="kuadrant-system/authorino-58cd955675-kq5jm"
Jul 24 14:39:39.370279 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.370252    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkpvj\" (UniqueName: \"kubernetes.io/projected/c1d48542-340c-417a-b7c7-c0a7c016582b-kube-api-access-fkpvj\") pod \"authorino-58cd955675-kq5jm\" (UID: \"c1d48542-340c-417a-b7c7-c0a7c016582b\") " pod="kuadrant-system/authorino-58cd955675-kq5jm"
Jul 24 14:39:39.390757 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.390731    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw" event={"ID":"745928f7-2edc-42d3-8be0-d3cbe0ba5988","Type":"ContainerStarted","Data":"eeb45ca2691b1c7ef23c5ac937b21a5bbd8ba8134dbd9326d3ef7b63ac765623"}
Jul 24 14:39:39.519188 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.519157    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kq5jm"
Jul 24 14:39:39.645420 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:39.645351    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kq5jm"]
Jul 24 14:39:39.648132 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:39:39.648104    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1d48542_340c_417a_b7c7_c0a7c016582b.slice/crio-ea2d7cc4fc7ef6bf00d29164c19f19f25eb77aa180bd248916ed0f4d2835b3df WatchSource:0}: Error finding container ea2d7cc4fc7ef6bf00d29164c19f19f25eb77aa180bd248916ed0f4d2835b3df: Status 404 returned error can't find the container with id ea2d7cc4fc7ef6bf00d29164c19f19f25eb77aa180bd248916ed0f4d2835b3df
Jul 24 14:39:40.400208 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:40.400170    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kq5jm" event={"ID":"c1d48542-340c-417a-b7c7-c0a7c016582b","Type":"ContainerStarted","Data":"ea2d7cc4fc7ef6bf00d29164c19f19f25eb77aa180bd248916ed0f4d2835b3df"}
Jul 24 14:39:43.412582 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:43.412555    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kq5jm"]
Jul 24 14:39:44.420115 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:44.420067    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw" event={"ID":"745928f7-2edc-42d3-8be0-d3cbe0ba5988","Type":"ContainerStarted","Data":"35871a29d712e05c1f18ac8e49d24ee676dc28c9af6c08cc9e438931539b40e0"}
Jul 24 14:39:44.420594 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:44.420159    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"
Jul 24 14:39:44.421526 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:44.421506    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kq5jm" event={"ID":"c1d48542-340c-417a-b7c7-c0a7c016582b","Type":"ContainerStarted","Data":"3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7"}
Jul 24 14:39:44.421631 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:44.421534    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-kq5jm" podUID="c1d48542-340c-417a-b7c7-c0a7c016582b" containerName="authorino" containerID="cri-o://3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7" gracePeriod=30
Jul 24 14:39:44.439851 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:44.439811    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw" podStartSLOduration=2.048421215 podStartE2EDuration="6.439796228s" totalImagesPullingTime="4.391375013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:39:44.438488038 +0000 UTC m=+462.942205995" watchObservedRunningTime="2026-07-24 14:39:44.439796228 +0000 UTC m=+462.943514209"
Jul 24 14:39:44.458032 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:44.457991    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-kq5jm" podStartSLOduration=1.466549672 podStartE2EDuration="5.45797807s" totalImagesPullingTime="3.991428398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:39:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:39:44.456505781 +0000 UTC m=+462.960223739" watchObservedRunningTime="2026-07-24 14:39:44.45797807 +0000 UTC m=+462.961696027"
Jul 24 14:39:44.671313 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:44.671252    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kq5jm"
Jul 24 14:39:44.807891 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:44.807843    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkpvj\" (UniqueName: \"kubernetes.io/projected/c1d48542-340c-417a-b7c7-c0a7c016582b-kube-api-access-fkpvj\") pod \"c1d48542-340c-417a-b7c7-c0a7c016582b\" (UID: \"c1d48542-340c-417a-b7c7-c0a7c016582b\") "
Jul 24 14:39:44.810030 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:44.809997    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1d48542-340c-417a-b7c7-c0a7c016582b-kube-api-access-fkpvj" (OuterVolumeSpecName: "kube-api-access-fkpvj") pod "c1d48542-340c-417a-b7c7-c0a7c016582b" (UID: "c1d48542-340c-417a-b7c7-c0a7c016582b"). InnerVolumeSpecName "kube-api-access-fkpvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:39:44.909028 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:44.908994    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkpvj\" (UniqueName: \"kubernetes.io/projected/c1d48542-340c-417a-b7c7-c0a7c016582b-kube-api-access-fkpvj\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:39:45.426828 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:45.426795    2581 generic.go:356] "Generic (PLEG): container finished" podID="c1d48542-340c-417a-b7c7-c0a7c016582b" containerID="3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7" exitCode=0
Jul 24 14:39:45.427296 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:45.426836    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kq5jm"
Jul 24 14:39:45.427296 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:45.426883    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kq5jm" event={"ID":"c1d48542-340c-417a-b7c7-c0a7c016582b","Type":"ContainerDied","Data":"3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7"}
Jul 24 14:39:45.427296 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:45.426927    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kq5jm" event={"ID":"c1d48542-340c-417a-b7c7-c0a7c016582b","Type":"ContainerDied","Data":"ea2d7cc4fc7ef6bf00d29164c19f19f25eb77aa180bd248916ed0f4d2835b3df"}
Jul 24 14:39:45.427296 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:45.426945    2581 scope.go:117] "RemoveContainer" containerID="3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7"
Jul 24 14:39:45.435085 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:45.435068    2581 scope.go:117] "RemoveContainer" containerID="3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7"
Jul 24 14:39:45.435339 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:39:45.435322    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7\": container with ID starting with 3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7 not found: ID does not exist" containerID="3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7"
Jul 24 14:39:45.435375 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:45.435347    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7"} err="failed to get container status \"3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7\": rpc error: code = NotFound desc = could not find container \"3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7\": container with ID starting with 3c82fa445fa062ed45025221b66ecda7406cfced512681dc7e3cb3de5d4360d7 not found: ID does not exist"
Jul 24 14:39:45.442594 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:45.442574    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kq5jm"]
Jul 24 14:39:45.446583 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:45.446561    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-kq5jm"]
Jul 24 14:39:47.334349 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:47.334320    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1d48542-340c-417a-b7c7-c0a7c016582b" path="/var/lib/kubelet/pods/c1d48542-340c-417a-b7c7-c0a7c016582b/volumes"
Jul 24 14:39:55.428217 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:55.428183    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"
Jul 24 14:39:59.603827 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:59.603784    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:39:59.717140 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:39:59.717077    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:00.073505 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:00.073465    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:12.679319 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:12.679284    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:12.903063 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:12.903025    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jgpfg"]
Jul 24 14:40:12.903421 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:12.903406    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1d48542-340c-417a-b7c7-c0a7c016582b" containerName="authorino"
Jul 24 14:40:12.903465 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:12.903424    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1d48542-340c-417a-b7c7-c0a7c016582b" containerName="authorino"
Jul 24 14:40:12.903499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:12.903477    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1d48542-340c-417a-b7c7-c0a7c016582b" containerName="authorino"
Jul 24 14:40:12.909470 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:12.909450    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jgpfg"
Jul 24 14:40:12.912473 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:12.912454    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xwk6h\""
Jul 24 14:40:12.915428 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:12.915406    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jgpfg"]
Jul 24 14:40:12.950998 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:12.950909    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk767\" (UniqueName: \"kubernetes.io/projected/ba1f9e04-5194-4b1c-9634-7cc6392d81a2-kube-api-access-fk767\") pod \"authorino-5d4b67c446-jgpfg\" (UID: \"ba1f9e04-5194-4b1c-9634-7cc6392d81a2\") " pod="kuadrant-system/authorino-5d4b67c446-jgpfg"
Jul 24 14:40:13.025713 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.025678    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:13.052217 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.052184    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk767\" (UniqueName: \"kubernetes.io/projected/ba1f9e04-5194-4b1c-9634-7cc6392d81a2-kube-api-access-fk767\") pod \"authorino-5d4b67c446-jgpfg\" (UID: \"ba1f9e04-5194-4b1c-9634-7cc6392d81a2\") " pod="kuadrant-system/authorino-5d4b67c446-jgpfg"
Jul 24 14:40:13.065175 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.065149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk767\" (UniqueName: \"kubernetes.io/projected/ba1f9e04-5194-4b1c-9634-7cc6392d81a2-kube-api-access-fk767\") pod \"authorino-5d4b67c446-jgpfg\" (UID: \"ba1f9e04-5194-4b1c-9634-7cc6392d81a2\") " pod="kuadrant-system/authorino-5d4b67c446-jgpfg"
Jul 24 14:40:13.146320 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.146283    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jgpfg"]
Jul 24 14:40:13.146533 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.146521    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jgpfg"
Jul 24 14:40:13.181945 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.181906    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-74df89b6c-25hkn"]
Jul 24 14:40:13.186135 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.186113    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74df89b6c-25hkn"
Jul 24 14:40:13.196731 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.196682    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74df89b6c-25hkn"]
Jul 24 14:40:13.253809 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.253781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mtps\" (UniqueName: \"kubernetes.io/projected/24f616d7-8678-49aa-b9b5-f781f86614b9-kube-api-access-8mtps\") pod \"authorino-74df89b6c-25hkn\" (UID: \"24f616d7-8678-49aa-b9b5-f781f86614b9\") " pod="kuadrant-system/authorino-74df89b6c-25hkn"
Jul 24 14:40:13.287499 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.287472    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jgpfg"]
Jul 24 14:40:13.290140 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:40:13.290115    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba1f9e04_5194_4b1c_9634_7cc6392d81a2.slice/crio-fb4fcae15aaacf99cb5a32117ee48e48b997e956143abc3e73fe7d1d2ca4f443 WatchSource:0}: Error finding container fb4fcae15aaacf99cb5a32117ee48e48b997e956143abc3e73fe7d1d2ca4f443: Status 404 returned error can't find the container with id fb4fcae15aaacf99cb5a32117ee48e48b997e956143abc3e73fe7d1d2ca4f443
Jul 24 14:40:13.354599 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.354560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mtps\" (UniqueName: \"kubernetes.io/projected/24f616d7-8678-49aa-b9b5-f781f86614b9-kube-api-access-8mtps\") pod \"authorino-74df89b6c-25hkn\" (UID: \"24f616d7-8678-49aa-b9b5-f781f86614b9\") " pod="kuadrant-system/authorino-74df89b6c-25hkn"
Jul 24 14:40:13.363430 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.363404    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mtps\" (UniqueName: \"kubernetes.io/projected/24f616d7-8678-49aa-b9b5-f781f86614b9-kube-api-access-8mtps\") pod \"authorino-74df89b6c-25hkn\" (UID: \"24f616d7-8678-49aa-b9b5-f781f86614b9\") " pod="kuadrant-system/authorino-74df89b6c-25hkn"
Jul 24 14:40:13.484208 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.484117    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74df89b6c-25hkn"]
Jul 24 14:40:13.484391 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.484379    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74df89b6c-25hkn"
Jul 24 14:40:13.552062 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.552028    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jgpfg" event={"ID":"ba1f9e04-5194-4b1c-9634-7cc6392d81a2","Type":"ContainerStarted","Data":"fb4fcae15aaacf99cb5a32117ee48e48b997e956143abc3e73fe7d1d2ca4f443"}
Jul 24 14:40:13.628604 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:13.628581    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74df89b6c-25hkn"]
Jul 24 14:40:13.630791 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:40:13.630766    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24f616d7_8678_49aa_b9b5_f781f86614b9.slice/crio-eacadd65266e79118a17eb5e94d253880cb27c334cd8a3e473e62a11ada9564b WatchSource:0}: Error finding container eacadd65266e79118a17eb5e94d253880cb27c334cd8a3e473e62a11ada9564b: Status 404 returned error can't find the container with id eacadd65266e79118a17eb5e94d253880cb27c334cd8a3e473e62a11ada9564b
Jul 24 14:40:14.558067 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.558029    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jgpfg" event={"ID":"ba1f9e04-5194-4b1c-9634-7cc6392d81a2","Type":"ContainerStarted","Data":"5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c"}
Jul 24 14:40:14.558556 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.558114    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-jgpfg" podUID="ba1f9e04-5194-4b1c-9634-7cc6392d81a2" containerName="authorino" containerID="cri-o://5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c" gracePeriod=30
Jul 24 14:40:14.559633 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.559613    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74df89b6c-25hkn" event={"ID":"24f616d7-8678-49aa-b9b5-f781f86614b9","Type":"ContainerStarted","Data":"a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12"}
Jul 24 14:40:14.559722 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.559642    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74df89b6c-25hkn" event={"ID":"24f616d7-8678-49aa-b9b5-f781f86614b9","Type":"ContainerStarted","Data":"eacadd65266e79118a17eb5e94d253880cb27c334cd8a3e473e62a11ada9564b"}
Jul 24 14:40:14.559722 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.559615    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-74df89b6c-25hkn" podUID="24f616d7-8678-49aa-b9b5-f781f86614b9" containerName="authorino" containerID="cri-o://a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12" gracePeriod=30
Jul 24 14:40:14.575306 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.575258    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-jgpfg" podStartSLOduration=2.265808913 podStartE2EDuration="2.575241818s" totalImagesPullingTime="309.432905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:40:14.574343288 +0000 UTC m=+493.078061249" watchObservedRunningTime="2026-07-24 14:40:14.575241818 +0000 UTC m=+493.078959835"
Jul 24 14:40:14.592744 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.592691    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-74df89b6c-25hkn" podStartSLOduration=1.17591752 podStartE2EDuration="1.592675183s" totalImagesPullingTime="416.757663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:40:14.591168165 +0000 UTC m=+493.094886124" watchObservedRunningTime="2026-07-24 14:40:14.592675183 +0000 UTC m=+493.096393140"
Jul 24 14:40:14.805115 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.805078    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jgpfg"
Jul 24 14:40:14.827290 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.827266    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74df89b6c-25hkn"
Jul 24 14:40:14.870530 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.870501    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk767\" (UniqueName: \"kubernetes.io/projected/ba1f9e04-5194-4b1c-9634-7cc6392d81a2-kube-api-access-fk767\") pod \"ba1f9e04-5194-4b1c-9634-7cc6392d81a2\" (UID: \"ba1f9e04-5194-4b1c-9634-7cc6392d81a2\") "
Jul 24 14:40:14.870707 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.870550    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mtps\" (UniqueName: \"kubernetes.io/projected/24f616d7-8678-49aa-b9b5-f781f86614b9-kube-api-access-8mtps\") pod \"24f616d7-8678-49aa-b9b5-f781f86614b9\" (UID: \"24f616d7-8678-49aa-b9b5-f781f86614b9\") "
Jul 24 14:40:14.872627 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.872603    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24f616d7-8678-49aa-b9b5-f781f86614b9-kube-api-access-8mtps" (OuterVolumeSpecName: "kube-api-access-8mtps") pod "24f616d7-8678-49aa-b9b5-f781f86614b9" (UID: "24f616d7-8678-49aa-b9b5-f781f86614b9"). InnerVolumeSpecName "kube-api-access-8mtps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:40:14.872735 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.872627    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba1f9e04-5194-4b1c-9634-7cc6392d81a2-kube-api-access-fk767" (OuterVolumeSpecName: "kube-api-access-fk767") pod "ba1f9e04-5194-4b1c-9634-7cc6392d81a2" (UID: "ba1f9e04-5194-4b1c-9634-7cc6392d81a2"). InnerVolumeSpecName "kube-api-access-fk767". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:40:14.971708 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.971675    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk767\" (UniqueName: \"kubernetes.io/projected/ba1f9e04-5194-4b1c-9634-7cc6392d81a2-kube-api-access-fk767\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:40:14.971708 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:14.971706    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mtps\" (UniqueName: \"kubernetes.io/projected/24f616d7-8678-49aa-b9b5-f781f86614b9-kube-api-access-8mtps\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:40:15.565206 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.565161    2581 generic.go:356] "Generic (PLEG): container finished" podID="ba1f9e04-5194-4b1c-9634-7cc6392d81a2" containerID="5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c" exitCode=0
Jul 24 14:40:15.565206 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.565202    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jgpfg"
Jul 24 14:40:15.565695 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.565250    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jgpfg" event={"ID":"ba1f9e04-5194-4b1c-9634-7cc6392d81a2","Type":"ContainerDied","Data":"5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c"}
Jul 24 14:40:15.565695 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.565307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jgpfg" event={"ID":"ba1f9e04-5194-4b1c-9634-7cc6392d81a2","Type":"ContainerDied","Data":"fb4fcae15aaacf99cb5a32117ee48e48b997e956143abc3e73fe7d1d2ca4f443"}
Jul 24 14:40:15.565695 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.565333    2581 scope.go:117] "RemoveContainer" containerID="5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c"
Jul 24 14:40:15.566643 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.566498    2581 generic.go:356] "Generic (PLEG): container finished" podID="24f616d7-8678-49aa-b9b5-f781f86614b9" containerID="a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12" exitCode=0
Jul 24 14:40:15.566643 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.566541    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74df89b6c-25hkn"
Jul 24 14:40:15.566643 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.566603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74df89b6c-25hkn" event={"ID":"24f616d7-8678-49aa-b9b5-f781f86614b9","Type":"ContainerDied","Data":"a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12"}
Jul 24 14:40:15.566643 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.566637    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74df89b6c-25hkn" event={"ID":"24f616d7-8678-49aa-b9b5-f781f86614b9","Type":"ContainerDied","Data":"eacadd65266e79118a17eb5e94d253880cb27c334cd8a3e473e62a11ada9564b"}
Jul 24 14:40:15.574037 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.574019    2581 scope.go:117] "RemoveContainer" containerID="5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c"
Jul 24 14:40:15.574316 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:40:15.574298    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c\": container with ID starting with 5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c not found: ID does not exist" containerID="5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c"
Jul 24 14:40:15.574367 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.574324    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c"} err="failed to get container status \"5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c\": rpc error: code = NotFound desc = could not find container \"5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c\": container with ID starting with 5574d20fbff010db5451e7d63002bbd2ad27b2155b512351d06adb077c758b6c not found: ID does not exist"
Jul 24 14:40:15.574367 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.574340    2581 scope.go:117] "RemoveContainer" containerID="a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12"
Jul 24 14:40:15.581733 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.581715    2581 scope.go:117] "RemoveContainer" containerID="a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12"
Jul 24 14:40:15.582006 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:40:15.581989    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12\": container with ID starting with a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12 not found: ID does not exist" containerID="a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12"
Jul 24 14:40:15.582059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.582014    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12"} err="failed to get container status \"a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12\": rpc error: code = NotFound desc = could not find container \"a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12\": container with ID starting with a33581d829f4bd549940588dd82f6f06aa6b411d1791c4d72ea10227e7f73f12 not found: ID does not exist"
Jul 24 14:40:15.585059 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.585038    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jgpfg"]
Jul 24 14:40:15.588566 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.588547    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jgpfg"]
Jul 24 14:40:15.599588 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.599523    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74df89b6c-25hkn"]
Jul 24 14:40:15.608348 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:15.608328    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-74df89b6c-25hkn"]
Jul 24 14:40:17.334956 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:17.334921    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24f616d7-8678-49aa-b9b5-f781f86614b9" path="/var/lib/kubelet/pods/24f616d7-8678-49aa-b9b5-f781f86614b9/volumes"
Jul 24 14:40:17.335438 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:17.335421    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba1f9e04-5194-4b1c-9634-7cc6392d81a2" path="/var/lib/kubelet/pods/ba1f9e04-5194-4b1c-9634-7cc6392d81a2/volumes"
Jul 24 14:40:21.230319 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:21.230276    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:21.320807 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:21.320768    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:21.394518 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:21.394484    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:21.467906 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:21.467879    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:24.417509 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:24.417471    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:24.523116 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:24.523069    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:57.223413 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:57.223323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:57.322495 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:57.322444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:57.729567 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:57.729527    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:58.025666 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:58.025577    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:58.796124 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:58.796075    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:58.941073 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:58.941040    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:40:59.015394 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:40:59.015361    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:41:07.670927 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:07.670885    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:07.777058 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:07.777011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:38.304773 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:38.304736    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:41:40.007318 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:40.007280    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:40.107955 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:40.107908    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:53.883933 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.883895    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"]
Jul 24 14:41:53.884380 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.884246    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba1f9e04-5194-4b1c-9634-7cc6392d81a2" containerName="authorino"
Jul 24 14:41:53.884380 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.884258    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba1f9e04-5194-4b1c-9634-7cc6392d81a2" containerName="authorino"
Jul 24 14:41:53.884380 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.884288    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24f616d7-8678-49aa-b9b5-f781f86614b9" containerName="authorino"
Jul 24 14:41:53.884380 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.884293    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="24f616d7-8678-49aa-b9b5-f781f86614b9" containerName="authorino"
Jul 24 14:41:53.884380 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.884346    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba1f9e04-5194-4b1c-9634-7cc6392d81a2" containerName="authorino"
Jul 24 14:41:53.884380 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.884352    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="24f616d7-8678-49aa-b9b5-f781f86614b9" containerName="authorino"
Jul 24 14:41:53.887256 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.887234    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:53.890883 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.890861    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gkqfh\""
Jul 24 14:41:53.890999 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.890861    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 14:41:53.890999 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.890915    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 14:41:53.891149 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.890927    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 24 14:41:53.898813 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.898791    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"]
Jul 24 14:41:53.908570 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.908546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:53.908664 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.908587    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:53.908705 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.908671    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:53.908739 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.908708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbdq5\" (UniqueName: \"kubernetes.io/projected/477a40a4-df3e-4fb0-8338-ddb4acb7274d-kube-api-access-pbdq5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:53.908773 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.908743    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:53.908773 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.908759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477a40a4-df3e-4fb0-8338-ddb4acb7274d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:53.913669 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:53.913639    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:41:54.009597 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.009566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.009597 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.009598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477a40a4-df3e-4fb0-8338-ddb4acb7274d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.009843 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.009623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.009843 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.009641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.009843 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.009701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.009843 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.009718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbdq5\" (UniqueName: \"kubernetes.io/projected/477a40a4-df3e-4fb0-8338-ddb4acb7274d-kube-api-access-pbdq5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.010050 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.010019    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.010050 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.010036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.010174 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.010088    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.012039 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.012011    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477a40a4-df3e-4fb0-8338-ddb4acb7274d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.012218 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.012199    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477a40a4-df3e-4fb0-8338-ddb4acb7274d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.018864 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.018841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbdq5\" (UniqueName: \"kubernetes.io/projected/477a40a4-df3e-4fb0-8338-ddb4acb7274d-kube-api-access-pbdq5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5hh4f\" (UID: \"477a40a4-df3e-4fb0-8338-ddb4acb7274d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.198272 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.198191    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:41:54.329496 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.329462    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"]
Jul 24 14:41:54.332652 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:41:54.332628    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477a40a4_df3e_4fb0_8338_ddb4acb7274d.slice/crio-f8391fac19b78fec3f022d42141f1615affa38d9157138ed05366d7b153e13db WatchSource:0}: Error finding container f8391fac19b78fec3f022d42141f1615affa38d9157138ed05366d7b153e13db: Status 404 returned error can't find the container with id f8391fac19b78fec3f022d42141f1615affa38d9157138ed05366d7b153e13db
Jul 24 14:41:54.609958 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:54.609931    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:41:55.018453 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:55.018371    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f" event={"ID":"477a40a4-df3e-4fb0-8338-ddb4acb7274d","Type":"ContainerStarted","Data":"f8391fac19b78fec3f022d42141f1615affa38d9157138ed05366d7b153e13db"}
Jul 24 14:41:56.023968 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:56.023932    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f" event={"ID":"477a40a4-df3e-4fb0-8338-ddb4acb7274d","Type":"ContainerStarted","Data":"4fdaa15c4b4abaeda5f93d82a6ba98d203169d5307d6595a827c2b628ee1cd32"}
Jul 24 14:41:56.048836 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:41:56.048780    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f" podStartSLOduration=1.891350412 podStartE2EDuration="3.04876508s" totalImagesPullingTime="1.157414668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:41:56.047655453 +0000 UTC m=+594.551373414" watchObservedRunningTime="2026-07-24 14:41:56.04876508 +0000 UTC m=+594.552483038"
Jul 24 14:42:04.198345 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:04.198292    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:42:04.200654 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:04.198370    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:42:04.211106 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:04.211067    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:42:05.074590 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:05.074561    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5hh4f"
Jul 24 14:42:14.006738 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:14.006699    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:42:15.459942 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:15.459907    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:25.716941 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:25.716832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:35.908424 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:35.908387    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:37.486227 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:37.486189    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:42:39.118140 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:39.118069    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:39.175476 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:39.175441    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:41.435010 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:41.434925    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:44.609605 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:44.609575    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:42:47.904584 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:47.904551    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"]
Jul 24 14:42:47.907911 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:47.907892    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:47.910668 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:47.910647    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 24 14:42:47.925742 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:47.925718    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"]
Jul 24 14:42:47.938450 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:47.938416    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:42:47.991208 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:47.991173    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:47.991208 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:47.991208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrv8d\" (UniqueName: \"kubernetes.io/projected/dcc9a946-328e-4497-b7e0-620008c93c73-kube-api-access-zrv8d\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:47.991402 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:47.991272    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcc9a946-328e-4497-b7e0-620008c93c73-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:47.991402 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:47.991302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:47.991469 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:47.991389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:47.991469 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:47.991423    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.092079 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.092041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcc9a946-328e-4497-b7e0-620008c93c73-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.092079 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.092107    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.092338 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.092168    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.092338 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.092196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.092338 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.092254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.092338 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.092276    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrv8d\" (UniqueName: \"kubernetes.io/projected/dcc9a946-328e-4497-b7e0-620008c93c73-kube-api-access-zrv8d\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.092630 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.092606    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.092700 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.092633    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.092700 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.092654    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.094507 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.094485    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcc9a946-328e-4497-b7e0-620008c93c73-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.094771 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.094754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcc9a946-328e-4497-b7e0-620008c93c73-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.100973 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.100954    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrv8d\" (UniqueName: \"kubernetes.io/projected/dcc9a946-328e-4497-b7e0-620008c93c73-kube-api-access-zrv8d\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v\" (UID: \"dcc9a946-328e-4497-b7e0-620008c93c73\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.216337 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.216243    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:48.355684 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.355652    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"]
Jul 24 14:42:48.358920 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:42:48.358887    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcc9a946_328e_4497_b7e0_620008c93c73.slice/crio-c8d6fd2323162cd8c193bb3ec53f4fc94a9ee6a6b85cedc56a8521203991b12a WatchSource:0}: Error finding container c8d6fd2323162cd8c193bb3ec53f4fc94a9ee6a6b85cedc56a8521203991b12a: Status 404 returned error can't find the container with id c8d6fd2323162cd8c193bb3ec53f4fc94a9ee6a6b85cedc56a8521203991b12a
Jul 24 14:42:48.360253 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:48.360237    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 14:42:49.256450 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:49.256412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v" event={"ID":"dcc9a946-328e-4497-b7e0-620008c93c73","Type":"ContainerStarted","Data":"5deafe02faf6fef52cc013d9c37648e852c21bfc6d4eb3bbbab0cab3e6c62523"}
Jul 24 14:42:49.256450 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:49.256455    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v" event={"ID":"dcc9a946-328e-4497-b7e0-620008c93c73","Type":"ContainerStarted","Data":"c8d6fd2323162cd8c193bb3ec53f4fc94a9ee6a6b85cedc56a8521203991b12a"}
Jul 24 14:42:49.277307 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:49.277257    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v" podStartSLOduration=1.8227365770000001 podStartE2EDuration="2.277241321s" totalImagesPullingTime="454.504744ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:42:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:42:49.27574544 +0000 UTC m=+647.779463411" watchObservedRunningTime="2026-07-24 14:42:49.277241321 +0000 UTC m=+647.780959280"
Jul 24 14:42:58.013077 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:58.013043    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:42:58.217298 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:58.217266    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:58.217450 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:58.217313    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:58.229415 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:58.229391    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:42:58.308693 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:42:58.308612    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v"
Jul 24 14:43:23.002731 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:43:23.002695    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:31.912232 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:43:31.912204    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:43.194720 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:43:43.194670    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:44.853165 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:43:44.853127    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:43:44.952942 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:43:44.952910    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:03.263651 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:44:03.263553    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:44:13.106108 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:44:13.106052    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:44:21.707417 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:44:21.707380    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:44:32.399430 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:44:32.399394    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:45:49.548937 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:45:49.548898    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:45:51.154234 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:45:51.154192    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:17.707857 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:17.707822    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:46:21.503644 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:21.503596    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:21.603489 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:21.603458    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:25.915847 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:25.915812    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:46:26.245167 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:26.245075    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:26.343070 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:26.343034    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:26.442145 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:26.442114    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:32.207955 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:32.207925    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:32.320863 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:32.320832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:32.530545 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:32.530450    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:32.629927 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:32.629896    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:38.612285 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:38.612250    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:38.753852 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:38.753813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:46:39.813145 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:39.813108    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:46:57.409937 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:46:57.409903    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:47:06.405765 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:47:06.405732    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:47:15.730897 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:47:15.730864    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:47:21.301161 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:47:21.301131    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:47:31.706589 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:47:31.706545    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:47:41.652178 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:47:41.652144    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:47:52.914224 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:47:52.914187    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:48:02.909019 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:48:02.908984    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:48:25.917816 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:48:25.917731    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:48:31.500999 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:48:31.500964    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:48:31.564183 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:48:31.564151    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:48:35.210523 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:48:35.210485    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:48:59.308954 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:48:59.308915    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:49:02.389671 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:49:02.389631    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:02.453616 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:49:02.453558    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:10.303451 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:49:10.303419    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:49:13.222715 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:49:13.222681    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:13.323508 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:49:13.323477    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:13.379161 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:49:13.379129    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:49:19.528843 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:49:19.528809    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:49:27.319846 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:49:27.319809    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:49:37.214452 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:49:37.214419    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:49:47.004451 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:49:47.004418    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:49:56.407548 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:49:56.407459    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:50:05.616320 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:50:05.616283    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:50:33.573754 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:50:33.573720    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:51:03.218764 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:51:03.218730    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:51:53.703993 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:51:53.703920    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:52:05.307109 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:52:05.307061    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:52:15.203598 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:52:15.203561    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:52:27.014046 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:52:27.014013    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:52:38.029815 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:52:38.029769    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:52:48.004550 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:52:48.004513    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:52:57.206555 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:52:57.206471    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:53:08.506292 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:53:08.506257    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:53:19.310638 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:53:19.310601    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:53:28.103231 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:53:28.103198    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:53:39.101940 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:53:39.101904    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:53:47.165871 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:53:47.165840    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:54:03.208533 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:03.208490    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:54:10.836509 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:10.836478    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:54:12.113693 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.113659    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"]
Jul 24 14:54:12.116979 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.116960    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.119867 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.119848    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-a94fbfb3-kserve-self-signed-certs\""
Jul 24 14:54:12.139650 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.139628    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"]
Jul 24 14:54:12.154207 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.154172    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:54:12.230992 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.230948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-dshm\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.231186 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.231003    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnxl6\" (UniqueName: \"kubernetes.io/projected/270c7bd5-e979-409b-b223-5b0c99c63113-kube-api-access-fnxl6\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.231186 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.231072    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-home\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.231186 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.231129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-model-cache\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.231186 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.231155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270c7bd5-e979-409b-b223-5b0c99c63113-tls-certs\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.231345 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.231190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-tmp-dir\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.331998 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.331967    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-dshm\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.331998 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.332004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnxl6\" (UniqueName: \"kubernetes.io/projected/270c7bd5-e979-409b-b223-5b0c99c63113-kube-api-access-fnxl6\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.332284 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.332038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-home\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.332284 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.332060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-model-cache\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.332284 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.332083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270c7bd5-e979-409b-b223-5b0c99c63113-tls-certs\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.332284 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.332142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-tmp-dir\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.332538 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.332515    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-model-cache\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.332599 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.332527    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-home\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.332599 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.332568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-tmp-dir\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.334374 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.334351    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-dshm\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.334512 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.334495    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270c7bd5-e979-409b-b223-5b0c99c63113-tls-certs\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.339939 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.339916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnxl6\" (UniqueName: \"kubernetes.io/projected/270c7bd5-e979-409b-b223-5b0c99c63113-kube-api-access-fnxl6\") pod \"e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") " pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.426729 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.426643    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:12.564920 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:12.564894    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"]
Jul 24 14:54:12.567003 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:54:12.566971    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod270c7bd5_e979_409b_b223_5b0c99c63113.slice/crio-ce31b68e407eae0a9b6b1755a892a58e8e1ec0d5cd31c7313a33d1c6d99b5113 WatchSource:0}: Error finding container ce31b68e407eae0a9b6b1755a892a58e8e1ec0d5cd31c7313a33d1c6d99b5113: Status 404 returned error can't find the container with id ce31b68e407eae0a9b6b1755a892a58e8e1ec0d5cd31c7313a33d1c6d99b5113
Jul 24 14:54:13.310176 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:13.310143    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p" event={"ID":"270c7bd5-e979-409b-b223-5b0c99c63113","Type":"ContainerStarted","Data":"41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53"}
Jul 24 14:54:13.310176 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:13.310182    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p" event={"ID":"270c7bd5-e979-409b-b223-5b0c99c63113","Type":"ContainerStarted","Data":"ce31b68e407eae0a9b6b1755a892a58e8e1ec0d5cd31c7313a33d1c6d99b5113"}
Jul 24 14:54:21.301160 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.301078    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p" podStartSLOduration=9.301060621 podStartE2EDuration="9.301060621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:54:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:54:13.344131422 +0000 UTC m=+1331.847849377" watchObservedRunningTime="2026-07-24 14:54:21.301060621 +0000 UTC m=+1339.804778578"
Jul 24 14:54:21.301682 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.301234    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"]
Jul 24 14:54:21.304699 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.304682    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.307436 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.307416    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-a94fbfb3-kserve-self-signed-certs\""
Jul 24 14:54:21.316904 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.316883    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"]
Jul 24 14:54:21.334039 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.334005    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:54:21.422468 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.422415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-home\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.422468 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.422484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-tmp-dir\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.422776 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.422523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-model-cache\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.422776 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.422623    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb8r6\" (UniqueName: \"kubernetes.io/projected/0434175f-2464-417c-9b44-508d1520e64e-kube-api-access-zb8r6\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.422776 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.422701    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-dshm\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.422776 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.422742    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0434175f-2464-417c-9b44-508d1520e64e-tls-certs\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.523976 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.523936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb8r6\" (UniqueName: \"kubernetes.io/projected/0434175f-2464-417c-9b44-508d1520e64e-kube-api-access-zb8r6\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.524200 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.523992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-dshm\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.524200 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.524025    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0434175f-2464-417c-9b44-508d1520e64e-tls-certs\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.524200 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.524069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-home\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.524200 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.524128    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-tmp-dir\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.524200 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.524179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-model-cache\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.524524 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.524504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-model-cache\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.524594 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.524571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-home\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.524631 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.524602    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-tmp-dir\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.526328 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.526310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-dshm\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.526590 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.526565    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0434175f-2464-417c-9b44-508d1520e64e-tls-certs\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.542956 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.542917    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb8r6\" (UniqueName: \"kubernetes.io/projected/0434175f-2464-417c-9b44-508d1520e64e-kube-api-access-zb8r6\") pod \"e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") " pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.614221 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.614180    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:21.745435 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:21.745328    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"]
Jul 24 14:54:21.747940 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:54:21.747910    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0434175f_2464_417c_9b44_508d1520e64e.slice/crio-cc5e030a94d040847e5a3c944715fa33eb78348091dfbc948be1040e3ca4ba7e WatchSource:0}: Error finding container cc5e030a94d040847e5a3c944715fa33eb78348091dfbc948be1040e3ca4ba7e: Status 404 returned error can't find the container with id cc5e030a94d040847e5a3c944715fa33eb78348091dfbc948be1040e3ca4ba7e
Jul 24 14:54:22.354380 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:22.354296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b" event={"ID":"0434175f-2464-417c-9b44-508d1520e64e","Type":"ContainerStarted","Data":"c907f1f2b766ebdd61d06526ee4571cee7397c78de2a8e30fb8aabdf11c892a3"}
Jul 24 14:54:22.354380 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:22.354343    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b" event={"ID":"0434175f-2464-417c-9b44-508d1520e64e","Type":"ContainerStarted","Data":"cc5e030a94d040847e5a3c944715fa33eb78348091dfbc948be1040e3ca4ba7e"}
Jul 24 14:54:22.375373 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:22.375327    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b" podStartSLOduration=1.375311892 podStartE2EDuration="1.375311892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:54:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:54:22.373749384 +0000 UTC m=+1340.877467342" watchObservedRunningTime="2026-07-24 14:54:22.375311892 +0000 UTC m=+1340.879029871"
Jul 24 14:54:22.427404 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:22.427365    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:22.427588 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:22.427426    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:22.439957 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:22.439927    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:23.371142 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:23.371046    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:54:24.504500 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:24.504465    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:54:31.614549 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:31.614503    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:31.615007 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:31.614565    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:31.627077 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:31.627043    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:32.412006 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:32.411979    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:54:46.700354 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:46.700319    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"]
Jul 24 14:54:46.750478 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:46.750448    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:54:48.391343 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:48.391310    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"]
Jul 24 14:54:48.415461 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:48.415423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:54:49.309212 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:54:49.309177    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:55:01.719347 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:01.719289    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p" podUID="270c7bd5-e979-409b-b223-5b0c99c63113" containerName="main" containerID="cri-o://41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53" gracePeriod=45
Jul 24 14:55:01.975352 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:01.975297    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:55:02.087684 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.087650    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-dshm\") pod \"270c7bd5-e979-409b-b223-5b0c99c63113\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") "
Jul 24 14:55:02.087857 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.087714    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnxl6\" (UniqueName: \"kubernetes.io/projected/270c7bd5-e979-409b-b223-5b0c99c63113-kube-api-access-fnxl6\") pod \"270c7bd5-e979-409b-b223-5b0c99c63113\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") "
Jul 24 14:55:02.087857 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.087732    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270c7bd5-e979-409b-b223-5b0c99c63113-tls-certs\") pod \"270c7bd5-e979-409b-b223-5b0c99c63113\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") "
Jul 24 14:55:02.087857 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.087776    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-model-cache\") pod \"270c7bd5-e979-409b-b223-5b0c99c63113\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") "
Jul 24 14:55:02.087857 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.087814    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-home\") pod \"270c7bd5-e979-409b-b223-5b0c99c63113\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") "
Jul 24 14:55:02.088071 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.087858    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-tmp-dir\") pod \"270c7bd5-e979-409b-b223-5b0c99c63113\" (UID: \"270c7bd5-e979-409b-b223-5b0c99c63113\") "
Jul 24 14:55:02.088174 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.088146    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-home" (OuterVolumeSpecName: "home") pod "270c7bd5-e979-409b-b223-5b0c99c63113" (UID: "270c7bd5-e979-409b-b223-5b0c99c63113"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:55:02.088232 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.088165    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-model-cache" (OuterVolumeSpecName: "model-cache") pod "270c7bd5-e979-409b-b223-5b0c99c63113" (UID: "270c7bd5-e979-409b-b223-5b0c99c63113"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:55:02.088232 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.088175    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "270c7bd5-e979-409b-b223-5b0c99c63113" (UID: "270c7bd5-e979-409b-b223-5b0c99c63113"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:55:02.089992 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.089969    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/270c7bd5-e979-409b-b223-5b0c99c63113-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "270c7bd5-e979-409b-b223-5b0c99c63113" (UID: "270c7bd5-e979-409b-b223-5b0c99c63113"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:55:02.089992 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.089983    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-dshm" (OuterVolumeSpecName: "dshm") pod "270c7bd5-e979-409b-b223-5b0c99c63113" (UID: "270c7bd5-e979-409b-b223-5b0c99c63113"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:55:02.090175 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.090016    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/270c7bd5-e979-409b-b223-5b0c99c63113-kube-api-access-fnxl6" (OuterVolumeSpecName: "kube-api-access-fnxl6") pod "270c7bd5-e979-409b-b223-5b0c99c63113" (UID: "270c7bd5-e979-409b-b223-5b0c99c63113"). InnerVolumeSpecName "kube-api-access-fnxl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:55:02.189551 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.189514    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:02.189551 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.189547    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:02.189551 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.189557    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnxl6\" (UniqueName: \"kubernetes.io/projected/270c7bd5-e979-409b-b223-5b0c99c63113-kube-api-access-fnxl6\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:02.189799 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.189565    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270c7bd5-e979-409b-b223-5b0c99c63113-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:02.189799 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.189575    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:02.189799 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.189582    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270c7bd5-e979-409b-b223-5b0c99c63113-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:02.542290 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.542250    2581 generic.go:356] "Generic (PLEG): container finished" podID="270c7bd5-e979-409b-b223-5b0c99c63113" containerID="41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53" exitCode=0
Jul 24 14:55:02.542489 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.542303    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"
Jul 24 14:55:02.542489 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.542319    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p" event={"ID":"270c7bd5-e979-409b-b223-5b0c99c63113","Type":"ContainerDied","Data":"41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53"}
Jul 24 14:55:02.542489 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.542355    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p" event={"ID":"270c7bd5-e979-409b-b223-5b0c99c63113","Type":"ContainerDied","Data":"ce31b68e407eae0a9b6b1755a892a58e8e1ec0d5cd31c7313a33d1c6d99b5113"}
Jul 24 14:55:02.542489 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.542372    2581 scope.go:117] "RemoveContainer" containerID="41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53"
Jul 24 14:55:02.551403 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.551383    2581 scope.go:117] "RemoveContainer" containerID="41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53"
Jul 24 14:55:02.551668 ip-10-0-143-246 kubenswrapper[2581]: E0724 14:55:02.551648    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53\": container with ID starting with 41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53 not found: ID does not exist" containerID="41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53"
Jul 24 14:55:02.551715 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.551678    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53"} err="failed to get container status \"41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53\": rpc error: code = NotFound desc = could not find container \"41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53\": container with ID starting with 41b5ba55843d1ee8a0ca9b146f7dc68a6500e4963e4e5141744f1eca0e291c53 not found: ID does not exist"
Jul 24 14:55:02.582302 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.582272    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"]
Jul 24 14:55:02.590531 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:02.590493    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-a94fbfb3-kserve-69f46fb4dc-sgw8p"]
Jul 24 14:55:03.335600 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.335567    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="270c7bd5-e979-409b-b223-5b0c99c63113" path="/var/lib/kubelet/pods/270c7bd5-e979-409b-b223-5b0c99c63113/volumes"
Jul 24 14:55:03.412167 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.412126    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b" podUID="0434175f-2464-417c-9b44-508d1520e64e" containerName="main" containerID="cri-o://c907f1f2b766ebdd61d06526ee4571cee7397c78de2a8e30fb8aabdf11c892a3" gracePeriod=45
Jul 24 14:55:03.549714 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.549683    2581 generic.go:356] "Generic (PLEG): container finished" podID="0434175f-2464-417c-9b44-508d1520e64e" containerID="c907f1f2b766ebdd61d06526ee4571cee7397c78de2a8e30fb8aabdf11c892a3" exitCode=0
Jul 24 14:55:03.549875 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.549752    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b" event={"ID":"0434175f-2464-417c-9b44-508d1520e64e","Type":"ContainerDied","Data":"c907f1f2b766ebdd61d06526ee4571cee7397c78de2a8e30fb8aabdf11c892a3"}
Jul 24 14:55:03.658543 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.658519    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:55:03.701368 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701342    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-home\") pod \"0434175f-2464-417c-9b44-508d1520e64e\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") "
Jul 24 14:55:03.701563 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701401    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0434175f-2464-417c-9b44-508d1520e64e-tls-certs\") pod \"0434175f-2464-417c-9b44-508d1520e64e\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") "
Jul 24 14:55:03.701563 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701456    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zb8r6\" (UniqueName: \"kubernetes.io/projected/0434175f-2464-417c-9b44-508d1520e64e-kube-api-access-zb8r6\") pod \"0434175f-2464-417c-9b44-508d1520e64e\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") "
Jul 24 14:55:03.701563 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701488    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-tmp-dir\") pod \"0434175f-2464-417c-9b44-508d1520e64e\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") "
Jul 24 14:55:03.701563 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701508    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-dshm\") pod \"0434175f-2464-417c-9b44-508d1520e64e\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") "
Jul 24 14:55:03.701563 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701550    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-model-cache\") pod \"0434175f-2464-417c-9b44-508d1520e64e\" (UID: \"0434175f-2464-417c-9b44-508d1520e64e\") "
Jul 24 14:55:03.701818 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701641    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-home" (OuterVolumeSpecName: "home") pod "0434175f-2464-417c-9b44-508d1520e64e" (UID: "0434175f-2464-417c-9b44-508d1520e64e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:55:03.701818 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701716    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0434175f-2464-417c-9b44-508d1520e64e" (UID: "0434175f-2464-417c-9b44-508d1520e64e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:55:03.701929 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701830    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-model-cache" (OuterVolumeSpecName: "model-cache") pod "0434175f-2464-417c-9b44-508d1520e64e" (UID: "0434175f-2464-417c-9b44-508d1520e64e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:55:03.701989 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701928    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:03.701989 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701948    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:03.701989 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.701957    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:03.703615 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.703588    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-dshm" (OuterVolumeSpecName: "dshm") pod "0434175f-2464-417c-9b44-508d1520e64e" (UID: "0434175f-2464-417c-9b44-508d1520e64e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:55:03.703615 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.703611    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0434175f-2464-417c-9b44-508d1520e64e-kube-api-access-zb8r6" (OuterVolumeSpecName: "kube-api-access-zb8r6") pod "0434175f-2464-417c-9b44-508d1520e64e" (UID: "0434175f-2464-417c-9b44-508d1520e64e"). InnerVolumeSpecName "kube-api-access-zb8r6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:55:03.703772 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.703670    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0434175f-2464-417c-9b44-508d1520e64e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0434175f-2464-417c-9b44-508d1520e64e" (UID: "0434175f-2464-417c-9b44-508d1520e64e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:55:03.802814 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.802782    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb8r6\" (UniqueName: \"kubernetes.io/projected/0434175f-2464-417c-9b44-508d1520e64e-kube-api-access-zb8r6\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:03.802814 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.802807    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0434175f-2464-417c-9b44-508d1520e64e-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:03.802814 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:03.802816    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0434175f-2464-417c-9b44-508d1520e64e-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:55:04.555531 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:04.555497    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b" event={"ID":"0434175f-2464-417c-9b44-508d1520e64e","Type":"ContainerDied","Data":"cc5e030a94d040847e5a3c944715fa33eb78348091dfbc948be1040e3ca4ba7e"}
Jul 24 14:55:04.555922 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:04.555535    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"
Jul 24 14:55:04.555922 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:04.555554    2581 scope.go:117] "RemoveContainer" containerID="c907f1f2b766ebdd61d06526ee4571cee7397c78de2a8e30fb8aabdf11c892a3"
Jul 24 14:55:04.575990 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:04.575961    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"]
Jul 24 14:55:04.578267 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:04.578245    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-a94fbfb3-kserve-d6857c74c-npt9b"]
Jul 24 14:55:05.104298 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:05.104261    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:55:05.334482 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:05.334445    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0434175f-2464-417c-9b44-508d1520e64e" path="/var/lib/kubelet/pods/0434175f-2464-417c-9b44-508d1520e64e/volumes"
Jul 24 14:55:12.111784 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:12.111744    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:55:28.419854 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:28.419818    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:55:39.806189 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:39.806156    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:55:49.207081 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:49.207047    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:55:57.509806 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:55:57.509726    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:56:08.528140 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:56:08.528102    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:56:18.396468 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:56:18.396433    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:56:31.971575 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:56:31.971541    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:56:47.918794 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:56:47.918757    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:56:59.992115 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:56:59.992062    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:57:13.108825 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:57:13.108786    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:57:21.507576 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:57:21.507541    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:57:24.901840 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:57:24.901762    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:57:34.607008 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:57:34.606975    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:57:46.516520 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:57:46.516485    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:57:59.135632 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:57:59.135598    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:58:02.519121 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:02.519059    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:08.409244 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:08.409202    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:58:10.640531 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:10.640501    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:10.742639 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:10.742602    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:26.181290 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:26.181252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:29.293697 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:29.293658    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:29.377703 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:29.377669    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:29.734946 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:29.734910    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:29.825846 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:29.825809    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:58:35.372007 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:35.371976    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:35.440003 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:35.439970    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:35.597070 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:35.597040    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:35.913678 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:35.913646    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:36.340973 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:36.340886    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:38.636215 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:38.636186    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:38.725854 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:38.725820    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:38.816701 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:38.816647    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:38.990103 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:38.989998    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:39.325682 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:39.325587    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:39.422994 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:39.422957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:58:47.114765 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:58:47.114728    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:59:04.171925 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.171844    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"]
Jul 24 14:59:04.172431 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.172245    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="270c7bd5-e979-409b-b223-5b0c99c63113" containerName="main"
Jul 24 14:59:04.172431 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.172258    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="270c7bd5-e979-409b-b223-5b0c99c63113" containerName="main"
Jul 24 14:59:04.172431 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.172267    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0434175f-2464-417c-9b44-508d1520e64e" containerName="main"
Jul 24 14:59:04.172431 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.172272    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0434175f-2464-417c-9b44-508d1520e64e" containerName="main"
Jul 24 14:59:04.172431 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.172325    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0434175f-2464-417c-9b44-508d1520e64e" containerName="main"
Jul 24 14:59:04.172431 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.172335    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="270c7bd5-e979-409b-b223-5b0c99c63113" containerName="main"
Jul 24 14:59:04.175043 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.175029    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"
Jul 24 14:59:04.177444 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.177424    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 14:59:04.177548 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.177460    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 14:59:04.182622 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.182393    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"]
Jul 24 14:59:04.195216 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.195190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc7ddf1c-0e31-4af8-aed1-febecbf297bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54\" (UID: \"dc7ddf1c-0e31-4af8-aed1-febecbf297bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"
Jul 24 14:59:04.295633 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.295589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc7ddf1c-0e31-4af8-aed1-febecbf297bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54\" (UID: \"dc7ddf1c-0e31-4af8-aed1-febecbf297bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"
Jul 24 14:59:04.296259 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.296234    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc7ddf1c-0e31-4af8-aed1-febecbf297bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54\" (UID: \"dc7ddf1c-0e31-4af8-aed1-febecbf297bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"
Jul 24 14:59:04.487111 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.487010    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"
Jul 24 14:59:04.617236 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.617212    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"]
Jul 24 14:59:04.619801 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:59:04.619774    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc7ddf1c_0e31_4af8_aed1_febecbf297bb.slice/crio-5da9174f4c2cdd6fc7ce59d41c796ad48973b0b1c7622f426652df73e753e438 WatchSource:0}: Error finding container 5da9174f4c2cdd6fc7ce59d41c796ad48973b0b1c7622f426652df73e753e438: Status 404 returned error can't find the container with id 5da9174f4c2cdd6fc7ce59d41c796ad48973b0b1c7622f426652df73e753e438
Jul 24 14:59:04.621674 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.621657    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 14:59:04.650563 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:04.650529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54" event={"ID":"dc7ddf1c-0e31-4af8-aed1-febecbf297bb","Type":"ContainerStarted","Data":"5da9174f4c2cdd6fc7ce59d41c796ad48973b0b1c7622f426652df73e753e438"}
Jul 24 14:59:07.666203 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:07.666175    2581 generic.go:356] "Generic (PLEG): container finished" podID="dc7ddf1c-0e31-4af8-aed1-febecbf297bb" containerID="f5f51736fd19374dc72f13d2971aed82d5898b79833505543abc8b458f17e9ff" exitCode=0
Jul 24 14:59:07.666536 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:07.666258    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54" event={"ID":"dc7ddf1c-0e31-4af8-aed1-febecbf297bb","Type":"ContainerDied","Data":"f5f51736fd19374dc72f13d2971aed82d5898b79833505543abc8b458f17e9ff"}
Jul 24 14:59:08.795453 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:08.795428    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"
Jul 24 14:59:08.828521 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:08.828496    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc7ddf1c-0e31-4af8-aed1-febecbf297bb-maas-api-service-ca\") pod \"dc7ddf1c-0e31-4af8-aed1-febecbf297bb\" (UID: \"dc7ddf1c-0e31-4af8-aed1-febecbf297bb\") "
Jul 24 14:59:08.828831 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:08.828809    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc7ddf1c-0e31-4af8-aed1-febecbf297bb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc7ddf1c-0e31-4af8-aed1-febecbf297bb" (UID: "dc7ddf1c-0e31-4af8-aed1-febecbf297bb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:59:08.929484 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:08.929403    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc7ddf1c-0e31-4af8-aed1-febecbf297bb-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:59:09.676624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:09.676589    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54" event={"ID":"dc7ddf1c-0e31-4af8-aed1-febecbf297bb","Type":"ContainerDied","Data":"5da9174f4c2cdd6fc7ce59d41c796ad48973b0b1c7622f426652df73e753e438"}
Jul 24 14:59:09.676624 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:09.676610    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"
Jul 24 14:59:09.676992 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:09.676636    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5da9174f4c2cdd6fc7ce59d41c796ad48973b0b1c7622f426652df73e753e438"
Jul 24 14:59:14.738867 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:14.738827    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:14.892486 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:14.892457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:14.990689 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:14.990593    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:15.390430 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:15.390391    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:15.524023 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:15.523957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:17.703722 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:17.703689    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:59:19.248354 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:19.248325    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:19.252576 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:19.252554    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"]
Jul 24 14:59:19.255005 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:19.254985    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-639b4a8c-kxk54"]
Jul 24 14:59:19.334195 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:19.334165    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc7ddf1c-0e31-4af8-aed1-febecbf297bb" path="/var/lib/kubelet/pods/dc7ddf1c-0e31-4af8-aed1-febecbf297bb/volumes"
Jul 24 14:59:19.556701 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:19.556620    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:19.737242 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:19.737211    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:21.055973 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:21.055940    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:21.156518 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:21.156491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:23.010130 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:23.010068    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:23.135832 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:23.135791    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:23.835465 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:23.835431    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:24.091425 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:24.091332    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:28.916983 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:28.916946    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 14:59:47.965733 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:47.965687    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"]
Jul 24 14:59:47.966188 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:47.966048    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc7ddf1c-0e31-4af8-aed1-febecbf297bb" containerName="revoke-keys"
Jul 24 14:59:47.966188 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:47.966059    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc7ddf1c-0e31-4af8-aed1-febecbf297bb" containerName="revoke-keys"
Jul 24 14:59:47.966188 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:47.966136    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc7ddf1c-0e31-4af8-aed1-febecbf297bb" containerName="revoke-keys"
Jul 24 14:59:47.973403 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:47.973383    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"
Jul 24 14:59:47.976379 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:47.976341    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 14:59:47.976552 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:47.976405    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 14:59:47.976552 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:47.976489    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"]
Jul 24 14:59:48.075134 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:48.075070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a00987-c10b-4ae2-8c30-c4c2435a5059-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5084c046-sqm25\" (UID: \"80a00987-c10b-4ae2-8c30-c4c2435a5059\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"
Jul 24 14:59:48.176024 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:48.175991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a00987-c10b-4ae2-8c30-c4c2435a5059-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5084c046-sqm25\" (UID: \"80a00987-c10b-4ae2-8c30-c4c2435a5059\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"
Jul 24 14:59:48.176655 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:48.176632    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a00987-c10b-4ae2-8c30-c4c2435a5059-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5084c046-sqm25\" (UID: \"80a00987-c10b-4ae2-8c30-c4c2435a5059\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"
Jul 24 14:59:48.286362 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:48.286285    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"
Jul 24 14:59:48.417615 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:48.417584    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"]
Jul 24 14:59:48.420724 ip-10-0-143-246 kubenswrapper[2581]: W0724 14:59:48.420694    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a00987_c10b_4ae2_8c30_c4c2435a5059.slice/crio-9644a6156f4fe6d7ab8dc527991d1d998aed8cb6ccc6f9b979c1e6982bdeb710 WatchSource:0}: Error finding container 9644a6156f4fe6d7ab8dc527991d1d998aed8cb6ccc6f9b979c1e6982bdeb710: Status 404 returned error can't find the container with id 9644a6156f4fe6d7ab8dc527991d1d998aed8cb6ccc6f9b979c1e6982bdeb710
Jul 24 14:59:48.853035 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:48.853002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25" event={"ID":"80a00987-c10b-4ae2-8c30-c4c2435a5059","Type":"ContainerStarted","Data":"928bcde24f0554eb8eecb774c1283101fa7b1aa24e0f1629b7a7542a00709b80"}
Jul 24 14:59:48.853208 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:48.853056    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25" event={"ID":"80a00987-c10b-4ae2-8c30-c4c2435a5059","Type":"ContainerStarted","Data":"9644a6156f4fe6d7ab8dc527991d1d998aed8cb6ccc6f9b979c1e6982bdeb710"}
Jul 24 14:59:48.869298 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:48.869252    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25" podStartSLOduration=1.869237574 podStartE2EDuration="1.869237574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:59:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:59:48.868479948 +0000 UTC m=+1667.372197919" watchObservedRunningTime="2026-07-24 14:59:48.869237574 +0000 UTC m=+1667.372955515"
Jul 24 14:59:49.858685 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:49.858650    2581 generic.go:356] "Generic (PLEG): container finished" podID="80a00987-c10b-4ae2-8c30-c4c2435a5059" containerID="928bcde24f0554eb8eecb774c1283101fa7b1aa24e0f1629b7a7542a00709b80" exitCode=0
Jul 24 14:59:49.859068 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:49.858713    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25" event={"ID":"80a00987-c10b-4ae2-8c30-c4c2435a5059","Type":"ContainerDied","Data":"928bcde24f0554eb8eecb774c1283101fa7b1aa24e0f1629b7a7542a00709b80"}
Jul 24 14:59:50.993451 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:50.993426    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"
Jul 24 14:59:51.103549 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:51.103502    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a00987-c10b-4ae2-8c30-c4c2435a5059-maas-api-service-ca\") pod \"80a00987-c10b-4ae2-8c30-c4c2435a5059\" (UID: \"80a00987-c10b-4ae2-8c30-c4c2435a5059\") "
Jul 24 14:59:51.103876 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:51.103851    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80a00987-c10b-4ae2-8c30-c4c2435a5059-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "80a00987-c10b-4ae2-8c30-c4c2435a5059" (UID: "80a00987-c10b-4ae2-8c30-c4c2435a5059"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:59:51.204314 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:51.204234    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a00987-c10b-4ae2-8c30-c4c2435a5059-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 14:59:51.869358 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:51.869326    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25" event={"ID":"80a00987-c10b-4ae2-8c30-c4c2435a5059","Type":"ContainerDied","Data":"9644a6156f4fe6d7ab8dc527991d1d998aed8cb6ccc6f9b979c1e6982bdeb710"}
Jul 24 14:59:51.869358 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:51.869355    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"
Jul 24 14:59:51.869566 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:51.869377    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9644a6156f4fe6d7ab8dc527991d1d998aed8cb6ccc6f9b979c1e6982bdeb710"
Jul 24 14:59:58.523212 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:58.523173    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:58.616202 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:58.616165    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 14:59:58.679481 ip-10-0-143-246 kubenswrapper[2581]: I0724 14:59:58.679449    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:00.147885 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.147854    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj"]
Jul 24 15:00:00.148308 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.148227    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a00987-c10b-4ae2-8c30-c4c2435a5059" containerName="revoke-keys"
Jul 24 15:00:00.148308 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.148238    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a00987-c10b-4ae2-8c30-c4c2435a5059" containerName="revoke-keys"
Jul 24 15:00:00.148308 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.148309    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="80a00987-c10b-4ae2-8c30-c4c2435a5059" containerName="revoke-keys"
Jul 24 15:00:00.151497 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.151481    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj"
Jul 24 15:00:00.153994 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.153974    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:00:00.160291 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.160264    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj"]
Jul 24 15:00:00.161800 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.161781    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj"
Jul 24 15:00:00.297839 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.297812    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj"]
Jul 24 15:00:00.299964 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:00:00.299936    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0cc1ba_2891_4a71_910c_bbbc89fc1243.slice/crio-bb9472ea379d0263a5c592351b4f81e5fb2472cfd5b9820cc69bae517f1c5d73 WatchSource:0}: Error finding container bb9472ea379d0263a5c592351b4f81e5fb2472cfd5b9820cc69bae517f1c5d73: Status 404 returned error can't find the container with id bb9472ea379d0263a5c592351b4f81e5fb2472cfd5b9820cc69bae517f1c5d73
Jul 24 15:00:00.911974 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.911928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj" event={"ID":"fc0cc1ba-2891-4a71-910c-bbbc89fc1243","Type":"ContainerStarted","Data":"0626017838bdfd63644883ea1fd535431da345183f698a244797cc53d549b80b"}
Jul 24 15:00:00.912174 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.912004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj" event={"ID":"fc0cc1ba-2891-4a71-910c-bbbc89fc1243","Type":"ContainerStarted","Data":"bb9472ea379d0263a5c592351b4f81e5fb2472cfd5b9820cc69bae517f1c5d73"}
Jul 24 15:00:00.928030 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:00.927983    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj" podStartSLOduration=0.927963005 podStartE2EDuration="927.963005ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:00:00.926464621 +0000 UTC m=+1679.430182582" watchObservedRunningTime="2026-07-24 15:00:00.927963005 +0000 UTC m=+1679.431680957"
Jul 24 15:00:01.641524 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:01.641487    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:00:01.918236 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:01.918149    2581 generic.go:356] "Generic (PLEG): container finished" podID="fc0cc1ba-2891-4a71-910c-bbbc89fc1243" containerID="0626017838bdfd63644883ea1fd535431da345183f698a244797cc53d549b80b" exitCode=0
Jul 24 15:00:01.918236 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:01.918197    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj" event={"ID":"fc0cc1ba-2891-4a71-910c-bbbc89fc1243","Type":"ContainerDied","Data":"0626017838bdfd63644883ea1fd535431da345183f698a244797cc53d549b80b"}
Jul 24 15:00:03.052514 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:03.052489    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj"
Jul 24 15:00:03.076392 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:03.076351    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"]
Jul 24 15:00:03.078486 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:03.078458    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5084c046-sqm25"]
Jul 24 15:00:03.289624 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:03.289540    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:03.334168 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:03.334136    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80a00987-c10b-4ae2-8c30-c4c2435a5059" path="/var/lib/kubelet/pods/80a00987-c10b-4ae2-8c30-c4c2435a5059/volumes"
Jul 24 15:00:03.367185 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:03.367148    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:03.825050 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:03.825019    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:03.928144 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:03.928110    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj"
Jul 24 15:00:03.928144 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:03.928138    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-wtbxj" event={"ID":"fc0cc1ba-2891-4a71-910c-bbbc89fc1243","Type":"ContainerDied","Data":"bb9472ea379d0263a5c592351b4f81e5fb2472cfd5b9820cc69bae517f1c5d73"}
Jul 24 15:00:03.928349 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:03.928182    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb9472ea379d0263a5c592351b4f81e5fb2472cfd5b9820cc69bae517f1c5d73"
Jul 24 15:00:03.954254 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:03.954222    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:04.113562 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:04.113528    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:04.456382 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:04.456305    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:06.123399 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:06.123369    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:06.226325 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:06.226287    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:06.518537 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:06.518457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:06.625500 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:06.625464    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:06.909017 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:06.908977    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:07.009468 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:07.009432    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:12.918926 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:12.918889    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:00:31.920431 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:31.920345    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"]
Jul 24 15:00:31.920879 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:31.920760    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0cc1ba-2891-4a71-910c-bbbc89fc1243" containerName="cleanup"
Jul 24 15:00:31.920879 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:31.920773    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0cc1ba-2891-4a71-910c-bbbc89fc1243" containerName="cleanup"
Jul 24 15:00:31.920879 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:31.920836    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc0cc1ba-2891-4a71-910c-bbbc89fc1243" containerName="cleanup"
Jul 24 15:00:31.925361 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:31.925345    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"
Jul 24 15:00:31.928232 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:31.928208    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:00:31.928361 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:31.928280    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:00:31.932885 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:31.932441    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"]
Jul 24 15:00:32.087523 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:32.087482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feff873c-94e3-4cf6-98b8-fa632d420ee5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s\" (UID: \"feff873c-94e3-4cf6-98b8-fa632d420ee5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"
Jul 24 15:00:32.189046 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:32.188954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feff873c-94e3-4cf6-98b8-fa632d420ee5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s\" (UID: \"feff873c-94e3-4cf6-98b8-fa632d420ee5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"
Jul 24 15:00:32.189637 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:32.189611    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feff873c-94e3-4cf6-98b8-fa632d420ee5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s\" (UID: \"feff873c-94e3-4cf6-98b8-fa632d420ee5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"
Jul 24 15:00:32.237130 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:32.237083    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"
Jul 24 15:00:32.367517 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:32.367493    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"]
Jul 24 15:00:32.369124 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:00:32.369072    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeff873c_94e3_4cf6_98b8_fa632d420ee5.slice/crio-63b558c821524a8f4e901c4d042b379890abb92e576afee7f00ea745c0cfb0b9 WatchSource:0}: Error finding container 63b558c821524a8f4e901c4d042b379890abb92e576afee7f00ea745c0cfb0b9: Status 404 returned error can't find the container with id 63b558c821524a8f4e901c4d042b379890abb92e576afee7f00ea745c0cfb0b9
Jul 24 15:00:33.060037 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:33.060004    2581 generic.go:356] "Generic (PLEG): container finished" podID="feff873c-94e3-4cf6-98b8-fa632d420ee5" containerID="48cb2e33831b47a762d974c75cfe665a851641ca3360e567558b8cc2e9cf69e4" exitCode=0
Jul 24 15:00:33.060390 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:33.060066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s" event={"ID":"feff873c-94e3-4cf6-98b8-fa632d420ee5","Type":"ContainerDied","Data":"48cb2e33831b47a762d974c75cfe665a851641ca3360e567558b8cc2e9cf69e4"}
Jul 24 15:00:33.060390 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:33.060129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s" event={"ID":"feff873c-94e3-4cf6-98b8-fa632d420ee5","Type":"ContainerStarted","Data":"63b558c821524a8f4e901c4d042b379890abb92e576afee7f00ea745c0cfb0b9"}
Jul 24 15:00:34.192503 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:34.192476    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"
Jul 24 15:00:34.312520 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:34.312486    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feff873c-94e3-4cf6-98b8-fa632d420ee5-maas-api-service-ca\") pod \"feff873c-94e3-4cf6-98b8-fa632d420ee5\" (UID: \"feff873c-94e3-4cf6-98b8-fa632d420ee5\") "
Jul 24 15:00:34.312866 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:34.312842    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feff873c-94e3-4cf6-98b8-fa632d420ee5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "feff873c-94e3-4cf6-98b8-fa632d420ee5" (UID: "feff873c-94e3-4cf6-98b8-fa632d420ee5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:00:34.413435 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:34.413402    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/feff873c-94e3-4cf6-98b8-fa632d420ee5-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:00:35.070843 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:35.070816    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"
Jul 24 15:00:35.071049 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:35.070849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s" event={"ID":"feff873c-94e3-4cf6-98b8-fa632d420ee5","Type":"ContainerDied","Data":"63b558c821524a8f4e901c4d042b379890abb92e576afee7f00ea745c0cfb0b9"}
Jul 24 15:00:35.071049 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:35.070889    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63b558c821524a8f4e901c4d042b379890abb92e576afee7f00ea745c0cfb0b9"
Jul 24 15:00:42.448080 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:42.448032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:42.536375 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:42.536340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:42.667324 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:42.667293    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:45.508038 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:45.508002    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:00:47.023874 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:47.023836    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"]
Jul 24 15:00:47.030187 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:47.030162    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7371ad-fnh7s"]
Jul 24 15:00:47.067630 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:47.067599    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:47.333683 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:47.333595    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feff873c-94e3-4cf6-98b8-fa632d420ee5" path="/var/lib/kubelet/pods/feff873c-94e3-4cf6-98b8-fa632d420ee5/volumes"
Jul 24 15:00:48.220559 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:48.220524    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:48.469210 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:48.469176    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:48.693111 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:48.693064    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:49.091339 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:49.091253    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:50.015955 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:50.015922    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:51.423988 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:51.423957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:51.543776 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:51.543736    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:51.898982 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:51.898940    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:52.014272 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:52.014237    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:52.230524 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:52.230439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:52.432675 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:52.432646    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:00:56.912674 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:00:56.912641    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:01:16.378086 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.378053    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"]
Jul 24 15:01:16.378713 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.378421    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feff873c-94e3-4cf6-98b8-fa632d420ee5" containerName="revoke-keys"
Jul 24 15:01:16.378713 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.378433    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="feff873c-94e3-4cf6-98b8-fa632d420ee5" containerName="revoke-keys"
Jul 24 15:01:16.378713 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.378514    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="feff873c-94e3-4cf6-98b8-fa632d420ee5" containerName="revoke-keys"
Jul 24 15:01:16.381491 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.381469    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"
Jul 24 15:01:16.384111 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.384074    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:01:16.384512 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.384487    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:01:16.392760 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.392738    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"]
Jul 24 15:01:16.480242 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.480204    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d295454d-2707-41dc-9511-a31bcaafc014-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4\" (UID: \"d295454d-2707-41dc-9511-a31bcaafc014\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"
Jul 24 15:01:16.580953 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.580905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d295454d-2707-41dc-9511-a31bcaafc014-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4\" (UID: \"d295454d-2707-41dc-9511-a31bcaafc014\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"
Jul 24 15:01:16.581637 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.581608    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d295454d-2707-41dc-9511-a31bcaafc014-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4\" (UID: \"d295454d-2707-41dc-9511-a31bcaafc014\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"
Jul 24 15:01:16.691815 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.691719    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"
Jul 24 15:01:16.832012 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:16.831975    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"]
Jul 24 15:01:16.833534 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:01:16.833507    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd295454d_2707_41dc_9511_a31bcaafc014.slice/crio-6dc41ce5696246243ac3b3e97e7c596d62ad31a27cf07a59ca01cf80f4ff9f69 WatchSource:0}: Error finding container 6dc41ce5696246243ac3b3e97e7c596d62ad31a27cf07a59ca01cf80f4ff9f69: Status 404 returned error can't find the container with id 6dc41ce5696246243ac3b3e97e7c596d62ad31a27cf07a59ca01cf80f4ff9f69
Jul 24 15:01:17.271866 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:17.271836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4" event={"ID":"d295454d-2707-41dc-9511-a31bcaafc014","Type":"ContainerStarted","Data":"1512d7c3ac50cbc1c6b3bb05bc754124a61b46f497564937beacfd76ec45cf3e"}
Jul 24 15:01:17.272023 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:17.271887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4" event={"ID":"d295454d-2707-41dc-9511-a31bcaafc014","Type":"ContainerStarted","Data":"6dc41ce5696246243ac3b3e97e7c596d62ad31a27cf07a59ca01cf80f4ff9f69"}
Jul 24 15:01:18.278883 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:18.278846    2581 generic.go:356] "Generic (PLEG): container finished" podID="d295454d-2707-41dc-9511-a31bcaafc014" containerID="1512d7c3ac50cbc1c6b3bb05bc754124a61b46f497564937beacfd76ec45cf3e" exitCode=0
Jul 24 15:01:18.279298 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:18.278934    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4" event={"ID":"d295454d-2707-41dc-9511-a31bcaafc014","Type":"ContainerDied","Data":"1512d7c3ac50cbc1c6b3bb05bc754124a61b46f497564937beacfd76ec45cf3e"}
Jul 24 15:01:19.411502 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:19.411478    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"
Jul 24 15:01:19.506165 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:19.506136    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d295454d-2707-41dc-9511-a31bcaafc014-maas-api-service-ca\") pod \"d295454d-2707-41dc-9511-a31bcaafc014\" (UID: \"d295454d-2707-41dc-9511-a31bcaafc014\") "
Jul 24 15:01:19.506464 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:19.506443    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d295454d-2707-41dc-9511-a31bcaafc014-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d295454d-2707-41dc-9511-a31bcaafc014" (UID: "d295454d-2707-41dc-9511-a31bcaafc014"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:01:19.607074 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:19.606979    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d295454d-2707-41dc-9511-a31bcaafc014-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:01:20.293281 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:20.293253    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"
Jul 24 15:01:20.293446 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:20.293252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4" event={"ID":"d295454d-2707-41dc-9511-a31bcaafc014","Type":"ContainerDied","Data":"6dc41ce5696246243ac3b3e97e7c596d62ad31a27cf07a59ca01cf80f4ff9f69"}
Jul 24 15:01:20.293446 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:20.293393    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6dc41ce5696246243ac3b3e97e7c596d62ad31a27cf07a59ca01cf80f4ff9f69"
Jul 24 15:01:27.186976 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:27.186938    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:27.317492 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:27.317460    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:27.411115 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:27.411066    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:30.463628 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:30.463595    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:01:31.540166 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:31.540123    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"]
Jul 24 15:01:31.543031 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:31.543003    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2feafa84-vfhz4"]
Jul 24 15:01:31.546954 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:31.546913    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:31.727150 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:31.727120    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:31.964699 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:31.964669    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:32.477583 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:32.477546    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:32.568850 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:32.568816    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:32.671448 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:32.671414    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:33.335408 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:33.335378    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d295454d-2707-41dc-9511-a31bcaafc014" path="/var/lib/kubelet/pods/d295454d-2707-41dc-9511-a31bcaafc014/volumes"
Jul 24 15:01:34.324246 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:34.324214    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:41.861484 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:41.861447    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:41.971315 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:41.971279    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:42.075694 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:42.075663    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:42.174071 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:42.173985    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:01:42.657918 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:01:42.657876    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:08.590763 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:02:08.590725    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:16.217838 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:02:16.217775    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:16.326400 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:02:16.326366    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:16.519236 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:02:16.519141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:32.327030 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:02:32.326994    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:39.666191 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:02:39.666152    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:39.865047 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:02:39.865011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:50.296982 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:02:50.296942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:02:51.100532 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:02:51.100499    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:01.026706 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:01.026667    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:01.213939 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:01.213903    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:01.320887 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:01.320797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:02.109653 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:02.109621    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:02.275625 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:02.275582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:04.169855 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:04.169819    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:06.828106 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:06.828061    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:03:10.821150 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:10.821112    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:10.928430 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:10.928367    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:11.537299 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:11.537264    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:17.160136 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:17.160082    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:03:18.353053 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:18.353017    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:18.445685 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:18.445651    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:18.625556 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:18.625516    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:19.184198 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:19.184168    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:19.481731 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:19.481642    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:22.086851 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:22.086815    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:22.344356 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:22.344272    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:22.776715 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:22.776680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:24.073409 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:24.073317    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:24.515289 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:24.515255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:26.564042 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:26.564011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:27.273410 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:27.273376    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:27.399356 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:27.399319    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:27.722615 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:27.722582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:29.819200 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:29.819168    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:03:33.983022 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:33.982973    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:34.086620 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.086581    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:34.477616 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.477583    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"]
Jul 24 15:03:34.477955 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.477942    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d295454d-2707-41dc-9511-a31bcaafc014" containerName="revoke-keys"
Jul 24 15:03:34.478017 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.477957    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d295454d-2707-41dc-9511-a31bcaafc014" containerName="revoke-keys"
Jul 24 15:03:34.478054 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.478024    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d295454d-2707-41dc-9511-a31bcaafc014" containerName="revoke-keys"
Jul 24 15:03:34.481171 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.481156    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"
Jul 24 15:03:34.485353 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.485335    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:03:34.485480 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.485409    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:03:34.556969 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.556933    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"]
Jul 24 15:03:34.586465 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.586431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c376ca04-62e1-4c0d-8db5-f7a612c9652f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6726d865-mvg6p\" (UID: \"c376ca04-62e1-4c0d-8db5-f7a612c9652f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"
Jul 24 15:03:34.687179 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.687144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c376ca04-62e1-4c0d-8db5-f7a612c9652f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6726d865-mvg6p\" (UID: \"c376ca04-62e1-4c0d-8db5-f7a612c9652f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"
Jul 24 15:03:34.687764 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.687739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c376ca04-62e1-4c0d-8db5-f7a612c9652f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6726d865-mvg6p\" (UID: \"c376ca04-62e1-4c0d-8db5-f7a612c9652f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"
Jul 24 15:03:34.791530 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.791436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"
Jul 24 15:03:34.941255 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.941222    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"]
Jul 24 15:03:34.945900 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:03:34.945872    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc376ca04_62e1_4c0d_8db5_f7a612c9652f.slice/crio-8a60d36daf6ec84ce12ccda982ce5f1f424df0aa10bf534118d9a421546b413c WatchSource:0}: Error finding container 8a60d36daf6ec84ce12ccda982ce5f1f424df0aa10bf534118d9a421546b413c: Status 404 returned error can't find the container with id 8a60d36daf6ec84ce12ccda982ce5f1f424df0aa10bf534118d9a421546b413c
Jul 24 15:03:34.979077 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:34.979047    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p" event={"ID":"c376ca04-62e1-4c0d-8db5-f7a612c9652f","Type":"ContainerStarted","Data":"8a60d36daf6ec84ce12ccda982ce5f1f424df0aa10bf534118d9a421546b413c"}
Jul 24 15:03:35.986235 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:35.986198    2581 generic.go:356] "Generic (PLEG): container finished" podID="c376ca04-62e1-4c0d-8db5-f7a612c9652f" containerID="5d6a272d608cc65efa508a42db5aeff282a543f3a8a04af33f1ac002897657ad" exitCode=0
Jul 24 15:03:35.986631 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:35.986245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p" event={"ID":"c376ca04-62e1-4c0d-8db5-f7a612c9652f","Type":"ContainerDied","Data":"5d6a272d608cc65efa508a42db5aeff282a543f3a8a04af33f1ac002897657ad"}
Jul 24 15:03:37.121589 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:37.121559    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"
Jul 24 15:03:37.209579 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:37.209549    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c376ca04-62e1-4c0d-8db5-f7a612c9652f-maas-api-service-ca\") pod \"c376ca04-62e1-4c0d-8db5-f7a612c9652f\" (UID: \"c376ca04-62e1-4c0d-8db5-f7a612c9652f\") "
Jul 24 15:03:37.209936 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:37.209914    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c376ca04-62e1-4c0d-8db5-f7a612c9652f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c376ca04-62e1-4c0d-8db5-f7a612c9652f" (UID: "c376ca04-62e1-4c0d-8db5-f7a612c9652f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:03:37.310833 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:37.310752    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c376ca04-62e1-4c0d-8db5-f7a612c9652f-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:03:38.001017 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:38.000985    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p" event={"ID":"c376ca04-62e1-4c0d-8db5-f7a612c9652f","Type":"ContainerDied","Data":"8a60d36daf6ec84ce12ccda982ce5f1f424df0aa10bf534118d9a421546b413c"}
Jul 24 15:03:38.001017 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:38.001013    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"
Jul 24 15:03:38.001290 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:38.001032    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a60d36daf6ec84ce12ccda982ce5f1f424df0aa10bf534118d9a421546b413c"
Jul 24 15:03:44.996421 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:44.996382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:45.090587 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:45.090557    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:45.192034 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:45.191994    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:48.015271 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:48.015242    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:03:49.493515 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:49.493477    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"]
Jul 24 15:03:49.496928 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:49.496903    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6726d865-mvg6p"]
Jul 24 15:03:49.500213 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:49.500183    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:49.923272 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:49.923236    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:51.334027 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:51.333996    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c376ca04-62e1-4c0d-8db5-f7a612c9652f" path="/var/lib/kubelet/pods/c376ca04-62e1-4c0d-8db5-f7a612c9652f/volumes"
Jul 24 15:03:56.085531 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:56.085500    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:56.186149 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:56.186113    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:56.517365 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:56.517327    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:56.695059 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:56.695029    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:56.844138 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:56.844024    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:56.937628 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:56.937590    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:03:57.061463 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:03:57.061428    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:02.922230 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:02.922193    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:03.027085 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:03.027052    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:03.126155 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:03.126120    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:03.294421 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:03.294332    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:09.886952 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:09.886889    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:09.997253 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:09.997221    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:16.782647 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:16.782613    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:17.127590 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:17.127558    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:17.386584 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:17.386480    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:20.126292 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:20.126252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:20.266209 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:20.266171    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:20.418937 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:20.418842    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:20.568014 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:20.567979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:21.228653 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:21.228618    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:21.324640 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:21.324606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:24.608456 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:24.608428    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:25.047403 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:25.047326    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:25.433786 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:25.433756    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:04:35.610784 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.610748    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"]
Jul 24 15:04:35.611318 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.611303    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c376ca04-62e1-4c0d-8db5-f7a612c9652f" containerName="revoke-keys"
Jul 24 15:04:35.611368 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.611322    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c376ca04-62e1-4c0d-8db5-f7a612c9652f" containerName="revoke-keys"
Jul 24 15:04:35.611424 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.611414    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c376ca04-62e1-4c0d-8db5-f7a612c9652f" containerName="revoke-keys"
Jul 24 15:04:35.614510 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.614493    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.617224 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.617200    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-040dfbba\"/\"e2e-lifecycle-model-040dfbba-kserve-self-signed-certs\""
Jul 24 15:04:35.618185 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.618166    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-040dfbba\"/\"openshift-service-ca.crt\""
Jul 24 15:04:35.618289 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.618226    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-040dfbba\"/\"default-dockercfg-q4ccx\""
Jul 24 15:04:35.618289 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.618174    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-040dfbba\"/\"kube-root-ca.crt\""
Jul 24 15:04:35.623737 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.623714    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"]
Jul 24 15:04:35.646203 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.646182    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:35.730661 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.730622    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ed465b-80c4-424a-b458-f166b23c5df3-tls-certs\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.730661 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.730665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-model-cache\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.730871 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.730694    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-dshm\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.730871 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.730755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb74c\" (UniqueName: \"kubernetes.io/projected/d0ed465b-80c4-424a-b458-f166b23c5df3-kube-api-access-fb74c\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.730871 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.730801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-tmp-dir\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.730871 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.730839    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-home\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.832077 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.832045    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-home\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.832305 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.832141    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ed465b-80c4-424a-b458-f166b23c5df3-tls-certs\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.832305 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.832164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-model-cache\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.832305 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.832198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-dshm\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.832479 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.832340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb74c\" (UniqueName: \"kubernetes.io/projected/d0ed465b-80c4-424a-b458-f166b23c5df3-kube-api-access-fb74c\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.832479 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.832392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-tmp-dir\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.832479 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.832469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-home\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.832645 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.832576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-model-cache\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.832786 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.832769    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-tmp-dir\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.834402 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.834387    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-dshm\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.834661 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.834642    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ed465b-80c4-424a-b458-f166b23c5df3-tls-certs\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.842625 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.842601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb74c\" (UniqueName: \"kubernetes.io/projected/d0ed465b-80c4-424a-b458-f166b23c5df3-kube-api-access-fb74c\") pod \"e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") " pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:35.924251 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:35.924160    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:36.060290 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:36.060265    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"]
Jul 24 15:04:36.063031 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:04:36.063002    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0ed465b_80c4_424a_b458_f166b23c5df3.slice/crio-0c8fc847105842387d0a66720a280370ec00f7a3552046cea150a11bfc58caf9 WatchSource:0}: Error finding container 0c8fc847105842387d0a66720a280370ec00f7a3552046cea150a11bfc58caf9: Status 404 returned error can't find the container with id 0c8fc847105842387d0a66720a280370ec00f7a3552046cea150a11bfc58caf9
Jul 24 15:04:36.272227 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:36.272128    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf" event={"ID":"d0ed465b-80c4-424a-b458-f166b23c5df3","Type":"ContainerStarted","Data":"948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64"}
Jul 24 15:04:36.272227 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:36.272174    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf" event={"ID":"d0ed465b-80c4-424a-b458-f166b23c5df3","Type":"ContainerStarted","Data":"0c8fc847105842387d0a66720a280370ec00f7a3552046cea150a11bfc58caf9"}
Jul 24 15:04:36.291584 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:36.291533    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf" podStartSLOduration=1.291518211 podStartE2EDuration="1.291518211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:04:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:04:36.289934653 +0000 UTC m=+1954.793652633" watchObservedRunningTime="2026-07-24 15:04:36.291518211 +0000 UTC m=+1954.795236170"
Jul 24 15:04:40.307692 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:40.307661    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:04:45.924440 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:45.924402    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:45.924840 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:45.924459    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:45.936876 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:45.936845    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:46.329751 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:46.329676    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:04:57.981634 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:57.981539    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:04:58.053147 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:04:58.053075    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:02.113744 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:02.113707    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:05:09.560975 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:09.560942    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"]
Jul 24 15:05:09.565019 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:09.564987    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"
Jul 24 15:05:09.568846 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:09.568816    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:05:09.569492 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:09.569474    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:05:09.576920 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:09.576887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"]
Jul 24 15:05:09.645212 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:09.645179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0767ba1a-2fd0-4323-9ad2-b04e35ff0e49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2\" (UID: \"0767ba1a-2fd0-4323-9ad2-b04e35ff0e49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"
Jul 24 15:05:09.746490 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:09.746459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0767ba1a-2fd0-4323-9ad2-b04e35ff0e49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2\" (UID: \"0767ba1a-2fd0-4323-9ad2-b04e35ff0e49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"
Jul 24 15:05:09.747052 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:09.747026    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0767ba1a-2fd0-4323-9ad2-b04e35ff0e49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2\" (UID: \"0767ba1a-2fd0-4323-9ad2-b04e35ff0e49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"
Jul 24 15:05:09.879399 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:09.879373    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"
Jul 24 15:05:10.007920 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:10.007893    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"]
Jul 24 15:05:10.012442 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:05:10.010179    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0767ba1a_2fd0_4323_9ad2_b04e35ff0e49.slice/crio-7eb99b2ab31555708c1d10f9f6471e833de4d0abcbb1122aab97893c42e23568 WatchSource:0}: Error finding container 7eb99b2ab31555708c1d10f9f6471e833de4d0abcbb1122aab97893c42e23568: Status 404 returned error can't find the container with id 7eb99b2ab31555708c1d10f9f6471e833de4d0abcbb1122aab97893c42e23568
Jul 24 15:05:10.438663 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:10.438627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2" event={"ID":"0767ba1a-2fd0-4323-9ad2-b04e35ff0e49","Type":"ContainerStarted","Data":"58c26507c01cb7638e01ba12d5252c151435612f281db383b32e559f625e2066"}
Jul 24 15:05:10.438819 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:10.438686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2" event={"ID":"0767ba1a-2fd0-4323-9ad2-b04e35ff0e49","Type":"ContainerStarted","Data":"7eb99b2ab31555708c1d10f9f6471e833de4d0abcbb1122aab97893c42e23568"}
Jul 24 15:05:10.459786 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:10.459738    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2" podStartSLOduration=1.459722256 podStartE2EDuration="1.459722256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:05:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:05:10.458892875 +0000 UTC m=+1988.962610832" watchObservedRunningTime="2026-07-24 15:05:10.459722256 +0000 UTC m=+1988.963440219"
Jul 24 15:05:11.444718 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:11.444686    2581 generic.go:356] "Generic (PLEG): container finished" podID="0767ba1a-2fd0-4323-9ad2-b04e35ff0e49" containerID="58c26507c01cb7638e01ba12d5252c151435612f281db383b32e559f625e2066" exitCode=0
Jul 24 15:05:11.445128 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:11.444728    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2" event={"ID":"0767ba1a-2fd0-4323-9ad2-b04e35ff0e49","Type":"ContainerDied","Data":"58c26507c01cb7638e01ba12d5252c151435612f281db383b32e559f625e2066"}
Jul 24 15:05:12.572910 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:12.572888    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"
Jul 24 15:05:12.672778 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:12.672747    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0767ba1a-2fd0-4323-9ad2-b04e35ff0e49-maas-api-service-ca\") pod \"0767ba1a-2fd0-4323-9ad2-b04e35ff0e49\" (UID: \"0767ba1a-2fd0-4323-9ad2-b04e35ff0e49\") "
Jul 24 15:05:12.673128 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:12.673081    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0767ba1a-2fd0-4323-9ad2-b04e35ff0e49-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0767ba1a-2fd0-4323-9ad2-b04e35ff0e49" (UID: "0767ba1a-2fd0-4323-9ad2-b04e35ff0e49"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:05:12.774363 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:12.774285    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0767ba1a-2fd0-4323-9ad2-b04e35ff0e49-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:05:13.455256 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:13.455223    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2" event={"ID":"0767ba1a-2fd0-4323-9ad2-b04e35ff0e49","Type":"ContainerDied","Data":"7eb99b2ab31555708c1d10f9f6471e833de4d0abcbb1122aab97893c42e23568"}
Jul 24 15:05:13.455421 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:13.455270    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7eb99b2ab31555708c1d10f9f6471e833de4d0abcbb1122aab97893c42e23568"
Jul 24 15:05:13.455421 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:13.455236    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"
Jul 24 15:05:23.724304 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:23.724271    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:05:25.195013 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:25.194979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:25.296010 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:25.295975    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:29.663751 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:29.663708    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:29.664269 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:29.664168    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"]
Jul 24 15:05:29.666451 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:29.666430    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-040dfbba-4c4c2"]
Jul 24 15:05:31.334544 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:31.334510    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0767ba1a-2fd0-4323-9ad2-b04e35ff0e49" path="/var/lib/kubelet/pods/0767ba1a-2fd0-4323-9ad2-b04e35ff0e49/volumes"
Jul 24 15:05:32.809841 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:32.809811    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:33.818244 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:33.818206    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:05:37.781532 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:37.781496    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"]
Jul 24 15:05:37.830448 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:37.830410    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:05:52.800465 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:52.800417    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf" podUID="d0ed465b-80c4-424a-b458-f166b23c5df3" containerName="main" containerID="cri-o://948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64" gracePeriod=45
Jul 24 15:05:53.050045 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.050017    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:05:53.137953 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.137924    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ed465b-80c4-424a-b458-f166b23c5df3-tls-certs\") pod \"d0ed465b-80c4-424a-b458-f166b23c5df3\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") "
Jul 24 15:05:53.138126 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.137977    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-model-cache\") pod \"d0ed465b-80c4-424a-b458-f166b23c5df3\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") "
Jul 24 15:05:53.138126 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.137994    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-dshm\") pod \"d0ed465b-80c4-424a-b458-f166b23c5df3\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") "
Jul 24 15:05:53.138126 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.138023    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-tmp-dir\") pod \"d0ed465b-80c4-424a-b458-f166b23c5df3\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") "
Jul 24 15:05:53.138126 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.138040    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-home\") pod \"d0ed465b-80c4-424a-b458-f166b23c5df3\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") "
Jul 24 15:05:53.138126 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.138088    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb74c\" (UniqueName: \"kubernetes.io/projected/d0ed465b-80c4-424a-b458-f166b23c5df3-kube-api-access-fb74c\") pod \"d0ed465b-80c4-424a-b458-f166b23c5df3\" (UID: \"d0ed465b-80c4-424a-b458-f166b23c5df3\") "
Jul 24 15:05:53.138378 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.138240    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0ed465b-80c4-424a-b458-f166b23c5df3" (UID: "d0ed465b-80c4-424a-b458-f166b23c5df3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:05:53.138378 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.138326    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-home" (OuterVolumeSpecName: "home") pod "d0ed465b-80c4-424a-b458-f166b23c5df3" (UID: "d0ed465b-80c4-424a-b458-f166b23c5df3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:05:53.138378 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.138361    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0ed465b-80c4-424a-b458-f166b23c5df3" (UID: "d0ed465b-80c4-424a-b458-f166b23c5df3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:05:53.138531 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.138400    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:05:53.138531 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.138411    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:05:53.140212 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.140191    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-dshm" (OuterVolumeSpecName: "dshm") pod "d0ed465b-80c4-424a-b458-f166b23c5df3" (UID: "d0ed465b-80c4-424a-b458-f166b23c5df3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:05:53.140290 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.140245    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ed465b-80c4-424a-b458-f166b23c5df3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0ed465b-80c4-424a-b458-f166b23c5df3" (UID: "d0ed465b-80c4-424a-b458-f166b23c5df3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:05:53.140351 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.140323    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ed465b-80c4-424a-b458-f166b23c5df3-kube-api-access-fb74c" (OuterVolumeSpecName: "kube-api-access-fb74c") pod "d0ed465b-80c4-424a-b458-f166b23c5df3" (UID: "d0ed465b-80c4-424a-b458-f166b23c5df3"). InnerVolumeSpecName "kube-api-access-fb74c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:05:53.239081 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.239045    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:05:53.239081 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.239074    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb74c\" (UniqueName: \"kubernetes.io/projected/d0ed465b-80c4-424a-b458-f166b23c5df3-kube-api-access-fb74c\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:05:53.239081 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.239083    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ed465b-80c4-424a-b458-f166b23c5df3-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:05:53.239081 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.239103    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ed465b-80c4-424a-b458-f166b23c5df3-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:05:53.651697 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.651662    2581 generic.go:356] "Generic (PLEG): container finished" podID="d0ed465b-80c4-424a-b458-f166b23c5df3" containerID="948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64" exitCode=0
Jul 24 15:05:53.651886 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.651707    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"
Jul 24 15:05:53.651886 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.651709    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf" event={"ID":"d0ed465b-80c4-424a-b458-f166b23c5df3","Type":"ContainerDied","Data":"948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64"}
Jul 24 15:05:53.651886 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.651746    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf" event={"ID":"d0ed465b-80c4-424a-b458-f166b23c5df3","Type":"ContainerDied","Data":"0c8fc847105842387d0a66720a280370ec00f7a3552046cea150a11bfc58caf9"}
Jul 24 15:05:53.651886 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.651767    2581 scope.go:117] "RemoveContainer" containerID="948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64"
Jul 24 15:05:53.660345 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.660324    2581 scope.go:117] "RemoveContainer" containerID="948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64"
Jul 24 15:05:53.660584 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:05:53.660563    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64\": container with ID starting with 948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64 not found: ID does not exist" containerID="948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64"
Jul 24 15:05:53.660658 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.660593    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64"} err="failed to get container status \"948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64\": rpc error: code = NotFound desc = could not find container \"948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64\": container with ID starting with 948f2c228425df5dc52983a3a67af3453946cab13bca687c45addba521aa1c64 not found: ID does not exist"
Jul 24 15:05:53.681828 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.681799    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"]
Jul 24 15:05:53.687342 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:53.687321    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-040dfbba/e2e-lifecycle-model-040dfbba-kserve-f85f9d89f-78rnf"]
Jul 24 15:05:55.336292 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:05:55.334830    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0ed465b-80c4-424a-b458-f166b23c5df3" path="/var/lib/kubelet/pods/d0ed465b-80c4-424a-b458-f166b23c5df3/volumes"
Jul 24 15:06:03.433710 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:03.433674    2581 scope.go:117] "RemoveContainer" containerID="928bcde24f0554eb8eecb774c1283101fa7b1aa24e0f1629b7a7542a00709b80"
Jul 24 15:06:03.441934 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:03.441913    2581 scope.go:117] "RemoveContainer" containerID="f5f51736fd19374dc72f13d2971aed82d5898b79833505543abc8b458f17e9ff"
Jul 24 15:06:14.216068 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:14.216023    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:15.775080 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:15.775041    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:17.141066 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:17.141031    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:17.630680 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:17.630644    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:21.512151 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:21.512074    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:21.613857 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:21.613812    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:21.805331 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:21.805245    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:22.044234 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:22.044202    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:23.155365 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:23.155333    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:23.253140 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:23.253105    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:23.353136 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:23.353084    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:24.196809 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:24.196775    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:24.293012 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:24.292962    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:24.393486 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:24.393451    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:25.904275 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:25.904232    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:31.753027 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:31.752995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:31.852477 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:31.852442    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:32.480979 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:32.480948    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:32.743328 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:32.743245    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:06:39.173110 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:39.173070    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:39.286713 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:39.286677    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:39.407290 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:39.407240    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:41.710688 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:41.710652    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:06:49.959817 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:49.959784    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:50.289888 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:50.289791    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:50.834212 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:50.834180    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:06:50.935806 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:06:50.935762    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:03.467038 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:03.467009    2581 scope.go:117] "RemoveContainer" containerID="48cb2e33831b47a762d974c75cfe665a851641ca3360e567558b8cc2e9cf69e4"
Jul 24 15:07:11.338003 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:11.337965    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:17.841327 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:17.841289    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:17.936745 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:17.936696    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:18.213425 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:18.213394    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:18.659273 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:18.659246    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:18.974694 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:18.974610    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:22.278190 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:22.278152    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:22.393334 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:22.393303    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:22.493637 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:22.493608    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:23.142126 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:23.142083    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:23.241767 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:23.241728    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:26.830894 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:26.830858    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:27.273779 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:27.273724    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:27.520531 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:27.520489    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:07:27.589142 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:27.589041    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:27.685611 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:27.685582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:30.258066 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:30.258017    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:30.352034 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:30.352001    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:30.437209 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:30.437148    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:30.542189 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:30.542105    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:30.934421 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:30.934390    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:31.034020 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:31.033981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:34.771358 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:34.771299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:41.031494 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.031458    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"]
Jul 24 15:07:41.031875 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.031836    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ed465b-80c4-424a-b458-f166b23c5df3" containerName="main"
Jul 24 15:07:41.031875 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.031847    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ed465b-80c4-424a-b458-f166b23c5df3" containerName="main"
Jul 24 15:07:41.031875 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.031865    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0767ba1a-2fd0-4323-9ad2-b04e35ff0e49" containerName="revoke-keys"
Jul 24 15:07:41.031875 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.031871    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0767ba1a-2fd0-4323-9ad2-b04e35ff0e49" containerName="revoke-keys"
Jul 24 15:07:41.032010 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.031920    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0767ba1a-2fd0-4323-9ad2-b04e35ff0e49" containerName="revoke-keys"
Jul 24 15:07:41.032010 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.031929    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ed465b-80c4-424a-b458-f166b23c5df3" containerName="main"
Jul 24 15:07:41.034672 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.034657    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.037865 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.037841    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-39743c\"/\"default-dockercfg-ndp4g\""
Jul 24 15:07:41.038656 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.038638    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-39743c\"/\"kube-root-ca.crt\""
Jul 24 15:07:41.039239 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.039220    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-39743c\"/\"test-model-39743c-kserve-self-signed-certs\""
Jul 24 15:07:41.039335 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.039239    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-39743c\"/\"openshift-service-ca.crt\""
Jul 24 15:07:41.047326 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.047298    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"]
Jul 24 15:07:41.055442 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.055415    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:07:41.084084 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.084053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-tmp-dir\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.084271 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.084117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-model-cache\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.084271 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.084179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-dshm\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.084271 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.084219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0204ee2-6135-4c98-acda-00584d5db22a-tls-certs\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.084271 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.084244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-home\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.084411 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.084272    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7d4q\" (UniqueName: \"kubernetes.io/projected/f0204ee2-6135-4c98-acda-00584d5db22a-kube-api-access-l7d4q\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.184730 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.184695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7d4q\" (UniqueName: \"kubernetes.io/projected/f0204ee2-6135-4c98-acda-00584d5db22a-kube-api-access-l7d4q\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.184914 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.184752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-tmp-dir\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.184914 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.184780    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-model-cache\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.184914 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.184833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-dshm\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.184914 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.184855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0204ee2-6135-4c98-acda-00584d5db22a-tls-certs\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.184914 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.184878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-home\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.185204 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.185113    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-tmp-dir\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.185285 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.185269    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-home\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.185346 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.185319    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-model-cache\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.187087 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.187066    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-dshm\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.187341 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.187325    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0204ee2-6135-4c98-acda-00584d5db22a-tls-certs\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.195542 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.195517    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7d4q\" (UniqueName: \"kubernetes.io/projected/f0204ee2-6135-4c98-acda-00584d5db22a-kube-api-access-l7d4q\") pod \"test-model-39743c-kserve-fb6f74d67-scq9b\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") " pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.344025 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.343935    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:41.478723 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:41.478698    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"]
Jul 24 15:07:41.480505 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:07:41.480470    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0204ee2_6135_4c98_acda_00584d5db22a.slice/crio-678e565c4cc031bbb74e04aac7b340e42a5ae2d27e49f190a08d3174cf413583 WatchSource:0}: Error finding container 678e565c4cc031bbb74e04aac7b340e42a5ae2d27e49f190a08d3174cf413583: Status 404 returned error can't find the container with id 678e565c4cc031bbb74e04aac7b340e42a5ae2d27e49f190a08d3174cf413583
Jul 24 15:07:42.161676 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:42.161641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b" event={"ID":"f0204ee2-6135-4c98-acda-00584d5db22a","Type":"ContainerStarted","Data":"8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f"}
Jul 24 15:07:42.161676 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:42.161681    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b" event={"ID":"f0204ee2-6135-4c98-acda-00584d5db22a","Type":"ContainerStarted","Data":"678e565c4cc031bbb74e04aac7b340e42a5ae2d27e49f190a08d3174cf413583"}
Jul 24 15:07:42.180489 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:42.180435    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b" podStartSLOduration=1.180418273 podStartE2EDuration="1.180418273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:07:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:07:42.179204513 +0000 UTC m=+2140.682922498" watchObservedRunningTime="2026-07-24 15:07:42.180418273 +0000 UTC m=+2140.684136230"
Jul 24 15:07:43.909040 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:43.909004    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:07:51.344690 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:51.344642    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:51.345145 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:51.344704    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:51.357407 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:51.357379    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:52.221442 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:52.221411    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:07:59.216606 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:07:59.216517    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:08:03.485840 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:03.485802    2581 scope.go:117] "RemoveContainer" containerID="1512d7c3ac50cbc1c6b3bb05bc754124a61b46f497564937beacfd76ec45cf3e"
Jul 24 15:08:16.919181 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:16.919151    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"]
Jul 24 15:08:16.922411 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:16.922391    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:16.926207 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:16.926184    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-97e60f\"/\"test-model-97e60f-kserve-self-signed-certs\""
Jul 24 15:08:16.926343 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:16.926213    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-97e60f\"/\"kube-root-ca.crt\""
Jul 24 15:08:16.926343 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:16.926301    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-97e60f\"/\"openshift-service-ca.crt\""
Jul 24 15:08:16.926343 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:16.926320    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-97e60f\"/\"default-dockercfg-m6wlw\""
Jul 24 15:08:16.936800 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:16.936777    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"]
Jul 24 15:08:16.947777 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:16.947752    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:08:17.010961 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.010925    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-home\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.010961 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.010963    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tmp-dir\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.011188 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.011070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tls-certs\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.011188 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.011116    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-model-cache\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.011188 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.011134    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qlvv\" (UniqueName: \"kubernetes.io/projected/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-kube-api-access-6qlvv\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.011290 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.011253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-dshm\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.112276 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.112239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-home\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.112276 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.112279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tmp-dir\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.112487 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.112326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tls-certs\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.112487 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.112347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-model-cache\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.112487 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.112362    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qlvv\" (UniqueName: \"kubernetes.io/projected/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-kube-api-access-6qlvv\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.112487 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.112457    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-dshm\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.112724 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.112697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-home\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.112787 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.112724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-model-cache\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.112869 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.112834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tmp-dir\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.114669 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.114648    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-dshm\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.114830 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.114813    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tls-certs\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.123407 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.123384    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qlvv\" (UniqueName: \"kubernetes.io/projected/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-kube-api-access-6qlvv\") pod \"test-model-97e60f-kserve-647f979f5-jtgr8\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") " pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.231426 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.231337    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:17.379419 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:17.379361    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"]
Jul 24 15:08:17.382915 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:08:17.382889    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod824f4f37_b43d_4a1f_9b5b_a9b07f1abd5d.slice/crio-57f0d10880bec6b06c85e3ddf2c2d98557924c004375690c56bb88ca630721fa WatchSource:0}: Error finding container 57f0d10880bec6b06c85e3ddf2c2d98557924c004375690c56bb88ca630721fa: Status 404 returned error can't find the container with id 57f0d10880bec6b06c85e3ddf2c2d98557924c004375690c56bb88ca630721fa
Jul 24 15:08:18.341371 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:18.341336    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8" event={"ID":"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d","Type":"ContainerStarted","Data":"0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8"}
Jul 24 15:08:18.341768 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:18.341380    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8" event={"ID":"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d","Type":"ContainerStarted","Data":"57f0d10880bec6b06c85e3ddf2c2d98557924c004375690c56bb88ca630721fa"}
Jul 24 15:08:18.360289 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:18.360245    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8" podStartSLOduration=2.360232824 podStartE2EDuration="2.360232824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:08:18.358981393 +0000 UTC m=+2176.862699364" watchObservedRunningTime="2026-07-24 15:08:18.360232824 +0000 UTC m=+2176.863950783"
Jul 24 15:08:21.824980 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:21.824938    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:08:27.231600 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:27.231570    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:27.232177 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:27.231624    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:27.245569 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:27.245545    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:08:27.392394 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:08:27.392367    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:09:05.111154 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:05.111120    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"]
Jul 24 15:09:05.181031 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:05.180998    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:06.049473 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.049440    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"]
Jul 24 15:09:06.072853 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.072825    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:06.202953 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.202915    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"]
Jul 24 15:09:06.206464 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.206447    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"
Jul 24 15:09:06.209374 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.209358    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:09:06.209465 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.209404    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:09:06.218017 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.217990    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"]
Jul 24 15:09:06.254879 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.254849    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15351f0a-d1db-42e1-9df8-7b85c1f99c18-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl\" (UID: \"15351f0a-d1db-42e1-9df8-7b85c1f99c18\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"
Jul 24 15:09:06.355965 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.355877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15351f0a-d1db-42e1-9df8-7b85c1f99c18-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl\" (UID: \"15351f0a-d1db-42e1-9df8-7b85c1f99c18\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"
Jul 24 15:09:06.356548 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.356527    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15351f0a-d1db-42e1-9df8-7b85c1f99c18-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl\" (UID: \"15351f0a-d1db-42e1-9df8-7b85c1f99c18\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"
Jul 24 15:09:06.516439 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.516403    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"
Jul 24 15:09:06.650647 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:06.650622    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"]
Jul 24 15:09:06.652822 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:09:06.652794    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15351f0a_d1db_42e1_9df8_7b85c1f99c18.slice/crio-960d9438bb105167ec0bf42431878c5fa97e12237e031eda2ace17a0aae28b04 WatchSource:0}: Error finding container 960d9438bb105167ec0bf42431878c5fa97e12237e031eda2ace17a0aae28b04: Status 404 returned error can't find the container with id 960d9438bb105167ec0bf42431878c5fa97e12237e031eda2ace17a0aae28b04
Jul 24 15:09:07.574002 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:07.573972    2581 generic.go:356] "Generic (PLEG): container finished" podID="15351f0a-d1db-42e1-9df8-7b85c1f99c18" containerID="d67422930f3ab43b5edd4bf5f8318f32691f1ab9d301cca326f7a0b01596e424" exitCode=0
Jul 24 15:09:07.574399 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:07.574025    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl" event={"ID":"15351f0a-d1db-42e1-9df8-7b85c1f99c18","Type":"ContainerDied","Data":"d67422930f3ab43b5edd4bf5f8318f32691f1ab9d301cca326f7a0b01596e424"}
Jul 24 15:09:07.574399 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:07.574076    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl" event={"ID":"15351f0a-d1db-42e1-9df8-7b85c1f99c18","Type":"ContainerStarted","Data":"960d9438bb105167ec0bf42431878c5fa97e12237e031eda2ace17a0aae28b04"}
Jul 24 15:09:08.707494 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:08.707473    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"
Jul 24 15:09:08.776926 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:08.776897    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15351f0a-d1db-42e1-9df8-7b85c1f99c18-maas-api-service-ca\") pod \"15351f0a-d1db-42e1-9df8-7b85c1f99c18\" (UID: \"15351f0a-d1db-42e1-9df8-7b85c1f99c18\") "
Jul 24 15:09:08.777288 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:08.777270    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15351f0a-d1db-42e1-9df8-7b85c1f99c18-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15351f0a-d1db-42e1-9df8-7b85c1f99c18" (UID: "15351f0a-d1db-42e1-9df8-7b85c1f99c18"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:09:08.877945 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:08.877919    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15351f0a-d1db-42e1-9df8-7b85c1f99c18-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:09.584134 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:09.584077    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl" event={"ID":"15351f0a-d1db-42e1-9df8-7b85c1f99c18","Type":"ContainerDied","Data":"960d9438bb105167ec0bf42431878c5fa97e12237e031eda2ace17a0aae28b04"}
Jul 24 15:09:09.584303 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:09.584146    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"
Jul 24 15:09:09.584303 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:09.584152    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="960d9438bb105167ec0bf42431878c5fa97e12237e031eda2ace17a0aae28b04"
Jul 24 15:09:09.817497 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:09.817464    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:09:16.644161 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:16.644122    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:16.714881 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:16.714845    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:16.793345 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:16.793300    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:20.129958 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.129921    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b" podUID="f0204ee2-6135-4c98-acda-00584d5db22a" containerName="main" containerID="cri-o://8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f" gracePeriod=45
Jul 24 15:09:20.383971 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.383913    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:09:20.481501 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481467    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-model-cache\") pod \"f0204ee2-6135-4c98-acda-00584d5db22a\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") "
Jul 24 15:09:20.481686 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481522    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-dshm\") pod \"f0204ee2-6135-4c98-acda-00584d5db22a\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") "
Jul 24 15:09:20.481686 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481571    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0204ee2-6135-4c98-acda-00584d5db22a-tls-certs\") pod \"f0204ee2-6135-4c98-acda-00584d5db22a\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") "
Jul 24 15:09:20.481686 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481593    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-tmp-dir\") pod \"f0204ee2-6135-4c98-acda-00584d5db22a\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") "
Jul 24 15:09:20.481686 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481610    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-home\") pod \"f0204ee2-6135-4c98-acda-00584d5db22a\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") "
Jul 24 15:09:20.481686 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481656    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7d4q\" (UniqueName: \"kubernetes.io/projected/f0204ee2-6135-4c98-acda-00584d5db22a-kube-api-access-l7d4q\") pod \"f0204ee2-6135-4c98-acda-00584d5db22a\" (UID: \"f0204ee2-6135-4c98-acda-00584d5db22a\") "
Jul 24 15:09:20.481947 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481797    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0204ee2-6135-4c98-acda-00584d5db22a" (UID: "f0204ee2-6135-4c98-acda-00584d5db22a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:09:20.481947 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481880    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0204ee2-6135-4c98-acda-00584d5db22a" (UID: "f0204ee2-6135-4c98-acda-00584d5db22a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:09:20.481947 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481890    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-home" (OuterVolumeSpecName: "home") pod "f0204ee2-6135-4c98-acda-00584d5db22a" (UID: "f0204ee2-6135-4c98-acda-00584d5db22a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:09:20.482058 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481968    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:20.482058 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481986    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:20.482058 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.481999    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:20.483819 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.483797    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-dshm" (OuterVolumeSpecName: "dshm") pod "f0204ee2-6135-4c98-acda-00584d5db22a" (UID: "f0204ee2-6135-4c98-acda-00584d5db22a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:09:20.483937 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.483869    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0204ee2-6135-4c98-acda-00584d5db22a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0204ee2-6135-4c98-acda-00584d5db22a" (UID: "f0204ee2-6135-4c98-acda-00584d5db22a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:09:20.483937 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.483872    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0204ee2-6135-4c98-acda-00584d5db22a-kube-api-access-l7d4q" (OuterVolumeSpecName: "kube-api-access-l7d4q") pod "f0204ee2-6135-4c98-acda-00584d5db22a" (UID: "f0204ee2-6135-4c98-acda-00584d5db22a"). InnerVolumeSpecName "kube-api-access-l7d4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:09:20.582884 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.582854    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7d4q\" (UniqueName: \"kubernetes.io/projected/f0204ee2-6135-4c98-acda-00584d5db22a-kube-api-access-l7d4q\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:20.582884 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.582882    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0204ee2-6135-4c98-acda-00584d5db22a-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:20.582884 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.582892    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0204ee2-6135-4c98-acda-00584d5db22a-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:20.642522 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.642429    2581 generic.go:356] "Generic (PLEG): container finished" podID="f0204ee2-6135-4c98-acda-00584d5db22a" containerID="8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f" exitCode=0
Jul 24 15:09:20.642522 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.642484    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"
Jul 24 15:09:20.642522 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.642504    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b" event={"ID":"f0204ee2-6135-4c98-acda-00584d5db22a","Type":"ContainerDied","Data":"8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f"}
Jul 24 15:09:20.642792 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.642537    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b" event={"ID":"f0204ee2-6135-4c98-acda-00584d5db22a","Type":"ContainerDied","Data":"678e565c4cc031bbb74e04aac7b340e42a5ae2d27e49f190a08d3174cf413583"}
Jul 24 15:09:20.642792 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.642559    2581 scope.go:117] "RemoveContainer" containerID="8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f"
Jul 24 15:09:20.651662 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.651641    2581 scope.go:117] "RemoveContainer" containerID="8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f"
Jul 24 15:09:20.651906 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:09:20.651887    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f\": container with ID starting with 8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f not found: ID does not exist" containerID="8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f"
Jul 24 15:09:20.651968 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.651915    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f"} err="failed to get container status \"8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f\": rpc error: code = NotFound desc = could not find container \"8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f\": container with ID starting with 8c8ca40cde6093a106c4bc0a7e66b300418986490af769ef26a03a013f3b4e0f not found: ID does not exist"
Jul 24 15:09:20.664906 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.664883    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"]
Jul 24 15:09:20.670567 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:20.670545    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-39743c/test-model-39743c-kserve-fb6f74d67-scq9b"]
Jul 24 15:09:21.068164 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.068069    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8" podUID="824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" containerName="main" containerID="cri-o://0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8" gracePeriod=45
Jul 24 15:09:21.278727 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.278694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:21.290192 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.290158    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"]
Jul 24 15:09:21.292564 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.292534    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39743c-b5nnl"]
Jul 24 15:09:21.316453 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.316435    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:09:21.334624 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.334569    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15351f0a-d1db-42e1-9df8-7b85c1f99c18" path="/var/lib/kubelet/pods/15351f0a-d1db-42e1-9df8-7b85c1f99c18/volumes"
Jul 24 15:09:21.335038 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.335019    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0204ee2-6135-4c98-acda-00584d5db22a" path="/var/lib/kubelet/pods/f0204ee2-6135-4c98-acda-00584d5db22a/volumes"
Jul 24 15:09:21.390305 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.390282    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tls-certs\") pod \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") "
Jul 24 15:09:21.390305 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.390309    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-dshm\") pod \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") "
Jul 24 15:09:21.390503 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.390341    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-home\") pod \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") "
Jul 24 15:09:21.390503 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.390362    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-model-cache\") pod \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") "
Jul 24 15:09:21.390503 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.390400    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qlvv\" (UniqueName: \"kubernetes.io/projected/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-kube-api-access-6qlvv\") pod \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") "
Jul 24 15:09:21.390503 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.390434    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tmp-dir\") pod \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\" (UID: \"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d\") "
Jul 24 15:09:21.390719 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.390605    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-home" (OuterVolumeSpecName: "home") pod "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" (UID: "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:09:21.390719 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.390673    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-model-cache" (OuterVolumeSpecName: "model-cache") pod "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" (UID: "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:09:21.390803 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.390715    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:21.390803 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.390716    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" (UID: "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:09:21.392436 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.392416    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-dshm" (OuterVolumeSpecName: "dshm") pod "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" (UID: "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:09:21.392531 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.392514    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" (UID: "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:09:21.392586 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.392525    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-kube-api-access-6qlvv" (OuterVolumeSpecName: "kube-api-access-6qlvv") pod "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" (UID: "824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d"). InnerVolumeSpecName "kube-api-access-6qlvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:09:21.491975 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.491946    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:21.491975 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.491973    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qlvv\" (UniqueName: \"kubernetes.io/projected/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-kube-api-access-6qlvv\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:21.492177 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.491984    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:21.492177 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.491995    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:21.492177 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.492003    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:21.650738 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.650704    2581 generic.go:356] "Generic (PLEG): container finished" podID="824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" containerID="0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8" exitCode=0
Jul 24 15:09:21.650912 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.650763    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"
Jul 24 15:09:21.650912 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.650790    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8" event={"ID":"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d","Type":"ContainerDied","Data":"0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8"}
Jul 24 15:09:21.650912 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.650833    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8" event={"ID":"824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d","Type":"ContainerDied","Data":"57f0d10880bec6b06c85e3ddf2c2d98557924c004375690c56bb88ca630721fa"}
Jul 24 15:09:21.650912 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.650857    2581 scope.go:117] "RemoveContainer" containerID="0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8"
Jul 24 15:09:21.659474 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.659458    2581 scope.go:117] "RemoveContainer" containerID="0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8"
Jul 24 15:09:21.659717 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:09:21.659699    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8\": container with ID starting with 0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8 not found: ID does not exist" containerID="0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8"
Jul 24 15:09:21.659757 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.659728    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8"} err="failed to get container status \"0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8\": rpc error: code = NotFound desc = could not find container \"0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8\": container with ID starting with 0c01894e322bcfe462078dd634a2d8be4fe9fea18ef938431a09e810c6c83ac8 not found: ID does not exist"
Jul 24 15:09:21.666845 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.666800    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:21.671620 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.671600    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"]
Jul 24 15:09:21.675305 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:21.675285    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-97e60f/test-model-97e60f-kserve-647f979f5-jtgr8"]
Jul 24 15:09:23.334676 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:23.334642    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" path="/var/lib/kubelet/pods/824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d/volumes"
Jul 24 15:09:25.814680 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:25.814646    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:09:26.101864 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:26.101772    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:27.824617 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:27.824565    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:27.895683 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:27.895650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:28.194677 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.194632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:28.362235 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.362196    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:28.482678 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.482600    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"]
Jul 24 15:09:28.483031 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.483014    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0204ee2-6135-4c98-acda-00584d5db22a" containerName="main"
Jul 24 15:09:28.483124 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.483034    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0204ee2-6135-4c98-acda-00584d5db22a" containerName="main"
Jul 24 15:09:28.483124 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.483047    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" containerName="main"
Jul 24 15:09:28.483124 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.483055    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" containerName="main"
Jul 24 15:09:28.483124 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.483065    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15351f0a-d1db-42e1-9df8-7b85c1f99c18" containerName="revoke-keys"
Jul 24 15:09:28.483124 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.483074    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="15351f0a-d1db-42e1-9df8-7b85c1f99c18" containerName="revoke-keys"
Jul 24 15:09:28.483386 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.483177    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0204ee2-6135-4c98-acda-00584d5db22a" containerName="main"
Jul 24 15:09:28.483386 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.483189    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="15351f0a-d1db-42e1-9df8-7b85c1f99c18" containerName="revoke-keys"
Jul 24 15:09:28.483386 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.483201    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="824f4f37-b43d-4a1f-9b5b-a9b07f1abd5d" containerName="main"
Jul 24 15:09:28.487328 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.487308    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"
Jul 24 15:09:28.490057 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.490034    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:09:28.490279 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.490260    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:09:28.495273 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.495247    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"]
Jul 24 15:09:28.657925 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.657893    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4083d81-7585-4cbb-ab2e-e71cb59de3a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv\" (UID: \"f4083d81-7585-4cbb-ab2e-e71cb59de3a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"
Jul 24 15:09:28.758654 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.758568    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4083d81-7585-4cbb-ab2e-e71cb59de3a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv\" (UID: \"f4083d81-7585-4cbb-ab2e-e71cb59de3a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"
Jul 24 15:09:28.759252 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.759231    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4083d81-7585-4cbb-ab2e-e71cb59de3a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv\" (UID: \"f4083d81-7585-4cbb-ab2e-e71cb59de3a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"
Jul 24 15:09:28.798546 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.798520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"
Jul 24 15:09:28.936040 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:28.936004    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"]
Jul 24 15:09:28.938459 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:09:28.938432    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4083d81_7585_4cbb_ab2e_e71cb59de3a0.slice/crio-cc7bfbd32d2054a559cd81fc900b62edd52d1404383930c4cde850da2c4aebb6 WatchSource:0}: Error finding container cc7bfbd32d2054a559cd81fc900b62edd52d1404383930c4cde850da2c4aebb6: Status 404 returned error can't find the container with id cc7bfbd32d2054a559cd81fc900b62edd52d1404383930c4cde850da2c4aebb6
Jul 24 15:09:29.691338 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:29.691311    2581 generic.go:356] "Generic (PLEG): container finished" podID="f4083d81-7585-4cbb-ab2e-e71cb59de3a0" containerID="6eade326bca8b124484fe4e0e05ed56b007342e151a6052f587a254ad3ad8c51" exitCode=0
Jul 24 15:09:29.691526 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:29.691391    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv" event={"ID":"f4083d81-7585-4cbb-ab2e-e71cb59de3a0","Type":"ContainerDied","Data":"6eade326bca8b124484fe4e0e05ed56b007342e151a6052f587a254ad3ad8c51"}
Jul 24 15:09:29.691526 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:29.691437    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv" event={"ID":"f4083d81-7585-4cbb-ab2e-e71cb59de3a0","Type":"ContainerStarted","Data":"cc7bfbd32d2054a559cd81fc900b62edd52d1404383930c4cde850da2c4aebb6"}
Jul 24 15:09:30.822107 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:30.822077    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"
Jul 24 15:09:30.977695 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:30.977612    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4083d81-7585-4cbb-ab2e-e71cb59de3a0-maas-api-service-ca\") pod \"f4083d81-7585-4cbb-ab2e-e71cb59de3a0\" (UID: \"f4083d81-7585-4cbb-ab2e-e71cb59de3a0\") "
Jul 24 15:09:30.977974 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:30.977952    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4083d81-7585-4cbb-ab2e-e71cb59de3a0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f4083d81-7585-4cbb-ab2e-e71cb59de3a0" (UID: "f4083d81-7585-4cbb-ab2e-e71cb59de3a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:09:31.079075 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:31.079044    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4083d81-7585-4cbb-ab2e-e71cb59de3a0-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:09:31.701273 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:31.701247    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"
Jul 24 15:09:31.701451 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:31.701292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv" event={"ID":"f4083d81-7585-4cbb-ab2e-e71cb59de3a0","Type":"ContainerDied","Data":"cc7bfbd32d2054a559cd81fc900b62edd52d1404383930c4cde850da2c4aebb6"}
Jul 24 15:09:31.701451 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:31.701360    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc7bfbd32d2054a559cd81fc900b62edd52d1404383930c4cde850da2c4aebb6"
Jul 24 15:09:38.958420 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:38.958381    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:39.069570 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:39.069523    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:41.800494 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:41.800454    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:09:43.552757 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:43.552727    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"]
Jul 24 15:09:43.555558 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:43.555531    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-97e60f-z8mwv"]
Jul 24 15:09:43.555708 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:43.555688    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:43.967236 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:43.967200    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:45.333610 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:45.333581    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4083d81-7585-4cbb-ab2e-e71cb59de3a0" path="/var/lib/kubelet/pods/f4083d81-7585-4cbb-ab2e-e71cb59de3a0/volumes"
Jul 24 15:09:50.080885 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:50.080848    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:50.181643 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:50.181599    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:50.404900 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:50.404870    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:50.629521 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:50.629485    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:51.007044 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:51.007005    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:51.330464 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:51.330376    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:51.667330 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:51.667303    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:54.006630 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:54.006595    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:54.106678 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:54.106644    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:54.222594 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:54.222561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:54.376310 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:54.376280    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:54.865980 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:54.865942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:54.961249 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:54.961212    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:58.527711 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:58.527677    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:59.024871 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:59.024837    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:59.370959 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:59.370919    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:09:59.735409 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:09:59.735324    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:01.914604 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:01.914569    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:10:02.142052 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:02.142019    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:02.244317 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:02.244236    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:02.338446 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:02.338412    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:02.572282 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:02.572184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:03.010807 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:03.010770    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:03.128559 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:03.128518    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:03.511538 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:03.511503    2581 scope.go:117] "RemoveContainer" containerID="5d6a272d608cc65efa508a42db5aeff282a543f3a8a04af33f1ac002897657ad"
Jul 24 15:10:06.668428 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:06.668398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:15.077156 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.077089    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"]
Jul 24 15:10:15.077538 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.077484    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4083d81-7585-4cbb-ab2e-e71cb59de3a0" containerName="revoke-keys"
Jul 24 15:10:15.077538 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.077495    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4083d81-7585-4cbb-ab2e-e71cb59de3a0" containerName="revoke-keys"
Jul 24 15:10:15.077616 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.077548    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4083d81-7585-4cbb-ab2e-e71cb59de3a0" containerName="revoke-keys"
Jul 24 15:10:15.080324 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.080308    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"
Jul 24 15:10:15.082873 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.082853    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:10:15.082993 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.082853    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:10:15.090280 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.090241    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"]
Jul 24 15:10:15.175939 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.175907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04e3b965-4496-42d1-b100-5b9a554adcf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9\" (UID: \"04e3b965-4496-42d1-b100-5b9a554adcf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"
Jul 24 15:10:15.277076 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.277041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04e3b965-4496-42d1-b100-5b9a554adcf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9\" (UID: \"04e3b965-4496-42d1-b100-5b9a554adcf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"
Jul 24 15:10:15.277682 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.277659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04e3b965-4496-42d1-b100-5b9a554adcf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9\" (UID: \"04e3b965-4496-42d1-b100-5b9a554adcf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"
Jul 24 15:10:15.390751 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.390725    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"
Jul 24 15:10:15.524063 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.524036    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"]
Jul 24 15:10:15.525823 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:10:15.525794    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e3b965_4496_42d1_b100_5b9a554adcf3.slice/crio-986bd616b4ff0896ff23e81e7a73485e008d1d35f4be7a81a90d23c32989d7b2 WatchSource:0}: Error finding container 986bd616b4ff0896ff23e81e7a73485e008d1d35f4be7a81a90d23c32989d7b2: Status 404 returned error can't find the container with id 986bd616b4ff0896ff23e81e7a73485e008d1d35f4be7a81a90d23c32989d7b2
Jul 24 15:10:15.915621 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.915586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9" event={"ID":"04e3b965-4496-42d1-b100-5b9a554adcf3","Type":"ContainerStarted","Data":"848a8874add967011bfa73d0f47dacae584057137a3f28ab902be58c35f0f18c"}
Jul 24 15:10:15.915818 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.915635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9" event={"ID":"04e3b965-4496-42d1-b100-5b9a554adcf3","Type":"ContainerStarted","Data":"986bd616b4ff0896ff23e81e7a73485e008d1d35f4be7a81a90d23c32989d7b2"}
Jul 24 15:10:15.936758 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:15.936708    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9" podStartSLOduration=0.936692119 podStartE2EDuration="936.692119ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:10:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:10:15.936163869 +0000 UTC m=+2294.439881827" watchObservedRunningTime="2026-07-24 15:10:15.936692119 +0000 UTC m=+2294.440410077"
Jul 24 15:10:16.922000 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:16.921964    2581 generic.go:356] "Generic (PLEG): container finished" podID="04e3b965-4496-42d1-b100-5b9a554adcf3" containerID="848a8874add967011bfa73d0f47dacae584057137a3f28ab902be58c35f0f18c" exitCode=0
Jul 24 15:10:16.922378 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:16.922045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9" event={"ID":"04e3b965-4496-42d1-b100-5b9a554adcf3","Type":"ContainerDied","Data":"848a8874add967011bfa73d0f47dacae584057137a3f28ab902be58c35f0f18c"}
Jul 24 15:10:18.057211 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:18.057188    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"
Jul 24 15:10:18.204862 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:18.204771    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04e3b965-4496-42d1-b100-5b9a554adcf3-maas-api-service-ca\") pod \"04e3b965-4496-42d1-b100-5b9a554adcf3\" (UID: \"04e3b965-4496-42d1-b100-5b9a554adcf3\") "
Jul 24 15:10:18.205162 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:18.205140    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04e3b965-4496-42d1-b100-5b9a554adcf3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "04e3b965-4496-42d1-b100-5b9a554adcf3" (UID: "04e3b965-4496-42d1-b100-5b9a554adcf3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:10:18.306478 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:18.306445    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04e3b965-4496-42d1-b100-5b9a554adcf3-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:10:18.932294 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:18.932260    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9" event={"ID":"04e3b965-4496-42d1-b100-5b9a554adcf3","Type":"ContainerDied","Data":"986bd616b4ff0896ff23e81e7a73485e008d1d35f4be7a81a90d23c32989d7b2"}
Jul 24 15:10:18.932473 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:18.932310    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="986bd616b4ff0896ff23e81e7a73485e008d1d35f4be7a81a90d23c32989d7b2"
Jul 24 15:10:18.932473 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:18.932271    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"
Jul 24 15:10:19.018834 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:19.018795    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:10:31.047238 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:31.047204    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:31.112358 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:31.112319    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:31.213946 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:31.213903    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:35.176608 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:35.176570    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"]
Jul 24 15:10:35.179632 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:35.179600    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b1e6f9-2dlc9"]
Jul 24 15:10:35.181013 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:35.180979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:35.212338 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:35.212296    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:10:35.333677 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:35.333644    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04e3b965-4496-42d1-b100-5b9a554adcf3" path="/var/lib/kubelet/pods/04e3b965-4496-42d1-b100-5b9a554adcf3/volumes"
Jul 24 15:10:35.592204 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:35.592105    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:42.520324 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:42.520289    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:42.673715 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:42.673683    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:42.904587 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:42.904556    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:43.093973 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.093933    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:43.209698 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.209610    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"]
Jul 24 15:10:43.210058 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.210039    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04e3b965-4496-42d1-b100-5b9a554adcf3" containerName="revoke-keys"
Jul 24 15:10:43.210168 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.210061    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="04e3b965-4496-42d1-b100-5b9a554adcf3" containerName="revoke-keys"
Jul 24 15:10:43.210235 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.210167    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="04e3b965-4496-42d1-b100-5b9a554adcf3" containerName="revoke-keys"
Jul 24 15:10:43.213211 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.213192    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"
Jul 24 15:10:43.215900 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.215879    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:10:43.216053 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.216036    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:10:43.222473 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.222451    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"]
Jul 24 15:10:43.317854 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.317809    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c560d05-b765-4403-bb8a-68e4cc005d37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b3c385-t77zc\" (UID: \"1c560d05-b765-4403-bb8a-68e4cc005d37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"
Jul 24 15:10:43.419333 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.419297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c560d05-b765-4403-bb8a-68e4cc005d37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b3c385-t77zc\" (UID: \"1c560d05-b765-4403-bb8a-68e4cc005d37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"
Jul 24 15:10:43.420030 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.420006    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c560d05-b765-4403-bb8a-68e4cc005d37-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b3c385-t77zc\" (UID: \"1c560d05-b765-4403-bb8a-68e4cc005d37\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"
Jul 24 15:10:43.524584 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.524489    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"
Jul 24 15:10:43.663179 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:43.663147    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"]
Jul 24 15:10:43.666487 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:10:43.666462    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c560d05_b765_4403_bb8a_68e4cc005d37.slice/crio-156c2f040d41d51c890d0b39321d9292fc951f769d19c273e07dc2c2503e6652 WatchSource:0}: Error finding container 156c2f040d41d51c890d0b39321d9292fc951f769d19c273e07dc2c2503e6652: Status 404 returned error can't find the container with id 156c2f040d41d51c890d0b39321d9292fc951f769d19c273e07dc2c2503e6652
Jul 24 15:10:44.049750 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:44.049711    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc" event={"ID":"1c560d05-b765-4403-bb8a-68e4cc005d37","Type":"ContainerStarted","Data":"66c67f428871598966d500bce73b14276c89e3306b7e979ea1c5f5d5b1dddaea"}
Jul 24 15:10:44.049920 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:44.049761    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc" event={"ID":"1c560d05-b765-4403-bb8a-68e4cc005d37","Type":"ContainerStarted","Data":"156c2f040d41d51c890d0b39321d9292fc951f769d19c273e07dc2c2503e6652"}
Jul 24 15:10:44.074141 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:44.074079    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc" podStartSLOduration=1.074064229 podStartE2EDuration="1.074064229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:10:44.071781024 +0000 UTC m=+2322.575498979" watchObservedRunningTime="2026-07-24 15:10:44.074064229 +0000 UTC m=+2322.577782248"
Jul 24 15:10:45.056878 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:45.056839    2581 generic.go:356] "Generic (PLEG): container finished" podID="1c560d05-b765-4403-bb8a-68e4cc005d37" containerID="66c67f428871598966d500bce73b14276c89e3306b7e979ea1c5f5d5b1dddaea" exitCode=0
Jul 24 15:10:45.057293 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:45.056926    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc" event={"ID":"1c560d05-b765-4403-bb8a-68e4cc005d37","Type":"ContainerDied","Data":"66c67f428871598966d500bce73b14276c89e3306b7e979ea1c5f5d5b1dddaea"}
Jul 24 15:10:46.194449 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:46.194426    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"
Jul 24 15:10:46.243336 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:46.243303    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c560d05-b765-4403-bb8a-68e4cc005d37-maas-api-service-ca\") pod \"1c560d05-b765-4403-bb8a-68e4cc005d37\" (UID: \"1c560d05-b765-4403-bb8a-68e4cc005d37\") "
Jul 24 15:10:46.243642 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:46.243620    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c560d05-b765-4403-bb8a-68e4cc005d37-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1c560d05-b765-4403-bb8a-68e4cc005d37" (UID: "1c560d05-b765-4403-bb8a-68e4cc005d37"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:10:46.344210 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:46.344126    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c560d05-b765-4403-bb8a-68e4cc005d37-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:10:47.067985 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:47.067948    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc" event={"ID":"1c560d05-b765-4403-bb8a-68e4cc005d37","Type":"ContainerDied","Data":"156c2f040d41d51c890d0b39321d9292fc951f769d19c273e07dc2c2503e6652"}
Jul 24 15:10:47.068209 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:47.068002    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="156c2f040d41d51c890d0b39321d9292fc951f769d19c273e07dc2c2503e6652"
Jul 24 15:10:47.068209 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:47.067964    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"
Jul 24 15:10:58.794304 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:58.794201    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:58.909282 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:58.909249    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:10:59.022961 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:10:59.022929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:03.207575 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:03.207539    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:11:03.281559 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:03.281523    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"]
Jul 24 15:11:03.285417 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:03.285371    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:03.287746 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:03.287716    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b3c385-t77zc"]
Jul 24 15:11:03.334429 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:03.334400    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c560d05-b765-4403-bb8a-68e4cc005d37" path="/var/lib/kubelet/pods/1c560d05-b765-4403-bb8a-68e4cc005d37/volumes"
Jul 24 15:11:03.742320 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:03.742284    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:09.675455 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:09.675400    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:09.776668 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:09.776630    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:09.981769 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:09.981675    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:10.305489 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:10.305399    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:10.632883 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:10.632844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:10.963920 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:10.963828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:11.704005 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:11.703964    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:14.546804 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:14.546767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:14.642178 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:14.642131    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:14.742158 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:14.742122    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:15.008957 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:15.008922    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:15.357783 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:15.357751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:15.454619 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:15.454566    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:19.135782 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:19.135743    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:19.534014 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:19.533924    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:19.924415 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:19.924375    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:20.157145 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:20.157110    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:22.631822 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:22.631787    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:11:22.681183 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:22.681149    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:22.827327 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:22.827295    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:23.660714 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:23.660679    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:24.231609 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:24.231579    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:24.329061 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:24.329028    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:24.433885 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:24.433843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:27.164482 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:27.164448    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:39.148306 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:39.148275    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:11:41.711441 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.711409    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"]
Jul 24 15:11:41.711847 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.711773    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c560d05-b765-4403-bb8a-68e4cc005d37" containerName="revoke-keys"
Jul 24 15:11:41.711847 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.711784    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c560d05-b765-4403-bb8a-68e4cc005d37" containerName="revoke-keys"
Jul 24 15:11:41.711926 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.711855    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c560d05-b765-4403-bb8a-68e4cc005d37" containerName="revoke-keys"
Jul 24 15:11:41.715068 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.715050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.718834 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.718776    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-dc7d81\"/\"auth-test-model-dc7d81-kserve-self-signed-certs\""
Jul 24 15:11:41.718834 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.718828    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-dc7d81\"/\"default-dockercfg-vtxpb\""
Jul 24 15:11:41.719124 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.719110    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-dc7d81\"/\"kube-root-ca.crt\""
Jul 24 15:11:41.719192 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.719158    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-dc7d81\"/\"openshift-service-ca.crt\""
Jul 24 15:11:41.730698 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.730669    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"]
Jul 24 15:11:41.754384 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.754353    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:11:41.854960 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.854924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-dshm\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.855160 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.854970    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-model-cache\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.855160 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.855006    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3868b2e-81d2-4aea-8f3d-300f80942777-tls-certs\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.855160 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.855026    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-tmp-dir\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.855160 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.855053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-home\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.855310 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.855196    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbbrf\" (UniqueName: \"kubernetes.io/projected/b3868b2e-81d2-4aea-8f3d-300f80942777-kube-api-access-gbbrf\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.956137 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.956077    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbbrf\" (UniqueName: \"kubernetes.io/projected/b3868b2e-81d2-4aea-8f3d-300f80942777-kube-api-access-gbbrf\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.956303 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.956164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-dshm\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.956303 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.956199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-model-cache\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.956303 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.956223    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3868b2e-81d2-4aea-8f3d-300f80942777-tls-certs\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.956303 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.956245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-tmp-dir\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.956303 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.956282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-home\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.956665 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.956638    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-model-cache\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.956757 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.956683    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-tmp-dir\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.956757 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.956730    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-home\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.958576 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.958558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-dshm\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.958839 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.958824    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3868b2e-81d2-4aea-8f3d-300f80942777-tls-certs\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:41.968591 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:41.968521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbbrf\" (UniqueName: \"kubernetes.io/projected/b3868b2e-81d2-4aea-8f3d-300f80942777-kube-api-access-gbbrf\") pod \"auth-test-model-dc7d81-kserve-54b7df776c-zb9w5\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") " pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:42.024586 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:42.024553    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:42.182206 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:42.182165    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"]
Jul 24 15:11:42.186058 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:11:42.186032    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3868b2e_81d2_4aea_8f3d_300f80942777.slice/crio-f4b7911ada1418d978e9fac26f3687662f506bd6d0f7e80dadf6a9dd93422cdf WatchSource:0}: Error finding container f4b7911ada1418d978e9fac26f3687662f506bd6d0f7e80dadf6a9dd93422cdf: Status 404 returned error can't find the container with id f4b7911ada1418d978e9fac26f3687662f506bd6d0f7e80dadf6a9dd93422cdf
Jul 24 15:11:42.333361 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:42.333328    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5" event={"ID":"b3868b2e-81d2-4aea-8f3d-300f80942777","Type":"ContainerStarted","Data":"d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337"}
Jul 24 15:11:42.333361 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:42.333367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5" event={"ID":"b3868b2e-81d2-4aea-8f3d-300f80942777","Type":"ContainerStarted","Data":"f4b7911ada1418d978e9fac26f3687662f506bd6d0f7e80dadf6a9dd93422cdf"}
Jul 24 15:11:42.354933 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:42.354881    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5" podStartSLOduration=1.354864436 podStartE2EDuration="1.354864436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:11:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:11:42.35213488 +0000 UTC m=+2380.855852830" watchObservedRunningTime="2026-07-24 15:11:42.354864436 +0000 UTC m=+2380.858582394"
Jul 24 15:11:52.025292 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:52.025251    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:52.025292 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:52.025300    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:52.037875 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:52.037848    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:52.396646 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:52.396612    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:11:53.629040 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:11:53.629007    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:12:03.546738 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:03.546709    2581 scope.go:117] "RemoveContainer" containerID="58c26507c01cb7638e01ba12d5252c151435612f281db383b32e559f625e2066"
Jul 24 15:12:09.420701 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.420663    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"]
Jul 24 15:12:09.423887 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.423865    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.426318 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.426301    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"auth-test-model-8ea489-kserve-self-signed-certs\""
Jul 24 15:12:09.427276 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.427258    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"kube-root-ca.crt\""
Jul 24 15:12:09.427378 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.427335    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"openshift-service-ca.crt\""
Jul 24 15:12:09.427449 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.427384    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"default-dockercfg-5c594\""
Jul 24 15:12:09.437403 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.437327    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"]
Jul 24 15:12:09.492294 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.492264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6ttk\" (UniqueName: \"kubernetes.io/projected/32298aab-a27a-4176-9c57-68b3f6c0fd41-kube-api-access-v6ttk\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.492294 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.492299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-dshm\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.492481 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.492324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-home\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.492481 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.492414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32298aab-a27a-4176-9c57-68b3f6c0fd41-tls-certs\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.492481 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.492447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-tmp-dir\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.492481 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.492477    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-model-cache\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.539965 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.539931    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:09.593157 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.593120    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32298aab-a27a-4176-9c57-68b3f6c0fd41-tls-certs\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.593333 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.593165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-tmp-dir\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.593333 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.593204    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-model-cache\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.593333 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.593300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6ttk\" (UniqueName: \"kubernetes.io/projected/32298aab-a27a-4176-9c57-68b3f6c0fd41-kube-api-access-v6ttk\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.593490 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.593332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-dshm\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.593490 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.593371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-home\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.593668 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.593645    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-tmp-dir\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.593737 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.593687    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-model-cache\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.593797 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.593769    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-home\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.595430 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.595411    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-dshm\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.595680 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.595665    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32298aab-a27a-4176-9c57-68b3f6c0fd41-tls-certs\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.603502 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.603480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6ttk\" (UniqueName: \"kubernetes.io/projected/32298aab-a27a-4176-9c57-68b3f6c0fd41-kube-api-access-v6ttk\") pod \"auth-test-model-8ea489-kserve-6946f4477f-6wdb4\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") " pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.732507 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.732411    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:09.863884 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:09.863853    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"]
Jul 24 15:12:09.865133 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:12:09.865085    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32298aab_a27a_4176_9c57_68b3f6c0fd41.slice/crio-e8d552608fda1611c60764f6d6a9af64af08aa9972d8437faa1a6a39f816a35a WatchSource:0}: Error finding container e8d552608fda1611c60764f6d6a9af64af08aa9972d8437faa1a6a39f816a35a: Status 404 returned error can't find the container with id e8d552608fda1611c60764f6d6a9af64af08aa9972d8437faa1a6a39f816a35a
Jul 24 15:12:10.475200 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:10.475165    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4" event={"ID":"32298aab-a27a-4176-9c57-68b3f6c0fd41","Type":"ContainerStarted","Data":"3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca"}
Jul 24 15:12:10.475658 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:10.475215    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4" event={"ID":"32298aab-a27a-4176-9c57-68b3f6c0fd41","Type":"ContainerStarted","Data":"e8d552608fda1611c60764f6d6a9af64af08aa9972d8437faa1a6a39f816a35a"}
Jul 24 15:12:10.494626 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:10.494575    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4" podStartSLOduration=1.494558743 podStartE2EDuration="1.494558743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:12:10.493161246 +0000 UTC m=+2408.996879204" watchObservedRunningTime="2026-07-24 15:12:10.494558743 +0000 UTC m=+2408.998276700"
Jul 24 15:12:13.201086 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:13.201051    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:12:19.732577 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:19.732541    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:19.732991 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:19.732682    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:19.745360 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:19.745338    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:20.535269 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:20.535246    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:12:31.916981 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:31.916950    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:12:43.194357 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:43.194315    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:12:49.413756 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:12:49.413720    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:13:06.539302 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:13:06.539268    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:13:23.409604 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:13:23.409564    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:13:39.019334 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:13:39.019300    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:13:42.418017 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:13:42.417986    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:13:43.813368 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:13:43.813331    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:13:55.106371 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:13:55.106281    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:13:55.209614 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:13:55.209571    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:13:57.562426 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:13:57.562389    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:14:14.742828 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:14.742793    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:14:21.194339 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:21.194305    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"]
Jul 24 15:14:21.231819 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:21.231790    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:14:22.090183 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:22.090149    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"]
Jul 24 15:14:22.112973 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:22.112942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:14:28.140584 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.140546    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"]
Jul 24 15:14:28.143769 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.143746    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.146536 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.146513    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-dc7d81\"/\"sub-test-model-dc7d81-kserve-self-signed-certs\""
Jul 24 15:14:28.155444 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.155416    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:14:28.156518 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.156497    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"]
Jul 24 15:14:28.234197 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.234165    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-home\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.234367 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.234217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-dshm\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.234367 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.234240    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-model-cache\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.234367 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.234286    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tls-certs\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.234367 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.234318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvb8s\" (UniqueName: \"kubernetes.io/projected/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-kube-api-access-pvb8s\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.234367 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.234344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tmp-dir\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.334902 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.334868    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-home\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.335122 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.334920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-dshm\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.335122 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.335046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-model-cache\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.335264 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.335132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tls-certs\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.335264 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.335166    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvb8s\" (UniqueName: \"kubernetes.io/projected/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-kube-api-access-pvb8s\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.335264 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.335202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tmp-dir\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.335418 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.335325    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-home\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.335418 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.335391    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-model-cache\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.335533 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.335499    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tmp-dir\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.337261 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.337240    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-dshm\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.337593 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.337573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tls-certs\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.343740 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.343714    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvb8s\" (UniqueName: \"kubernetes.io/projected/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-kube-api-access-pvb8s\") pod \"sub-test-model-dc7d81-kserve-7c446d7689-skpwf\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") " pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.453723 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.453642    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:28.600011 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:28.599987    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"]
Jul 24 15:14:28.602606 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:14:28.602580    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a5ec87c_b71f_48bd_ab87_06bb330fd3b1.slice/crio-448c7d0ba0cb5a4a9217a4034d8d077af44008116c52344818c64509cb738bb8 WatchSource:0}: Error finding container 448c7d0ba0cb5a4a9217a4034d8d077af44008116c52344818c64509cb738bb8: Status 404 returned error can't find the container with id 448c7d0ba0cb5a4a9217a4034d8d077af44008116c52344818c64509cb738bb8
Jul 24 15:14:29.132637 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:29.132599    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf" event={"ID":"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1","Type":"ContainerStarted","Data":"913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b"}
Jul 24 15:14:29.132637 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:29.132639    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf" event={"ID":"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1","Type":"ContainerStarted","Data":"448c7d0ba0cb5a4a9217a4034d8d077af44008116c52344818c64509cb738bb8"}
Jul 24 15:14:29.155702 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:29.155650    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf" podStartSLOduration=1.155636902 podStartE2EDuration="1.155636902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:14:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:14:29.154542407 +0000 UTC m=+2547.658260365" watchObservedRunningTime="2026-07-24 15:14:29.155636902 +0000 UTC m=+2547.659354860"
Jul 24 15:14:33.924627 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:33.924592    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:14:36.213083 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.213041    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5" podUID="b3868b2e-81d2-4aea-8f3d-300f80942777" containerName="main" containerID="cri-o://d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337" gracePeriod=45
Jul 24 15:14:36.474540 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.474517    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:14:36.506322 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506295    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-model-cache\") pod \"b3868b2e-81d2-4aea-8f3d-300f80942777\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") "
Jul 24 15:14:36.506475 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506374    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-tmp-dir\") pod \"b3868b2e-81d2-4aea-8f3d-300f80942777\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") "
Jul 24 15:14:36.506475 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506402    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-home\") pod \"b3868b2e-81d2-4aea-8f3d-300f80942777\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") "
Jul 24 15:14:36.506475 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506428    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbbrf\" (UniqueName: \"kubernetes.io/projected/b3868b2e-81d2-4aea-8f3d-300f80942777-kube-api-access-gbbrf\") pod \"b3868b2e-81d2-4aea-8f3d-300f80942777\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") "
Jul 24 15:14:36.506634 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506482    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3868b2e-81d2-4aea-8f3d-300f80942777-tls-certs\") pod \"b3868b2e-81d2-4aea-8f3d-300f80942777\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") "
Jul 24 15:14:36.506634 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506536    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-dshm\") pod \"b3868b2e-81d2-4aea-8f3d-300f80942777\" (UID: \"b3868b2e-81d2-4aea-8f3d-300f80942777\") "
Jul 24 15:14:36.506634 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506576    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-model-cache" (OuterVolumeSpecName: "model-cache") pod "b3868b2e-81d2-4aea-8f3d-300f80942777" (UID: "b3868b2e-81d2-4aea-8f3d-300f80942777"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:36.506791 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506673    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b3868b2e-81d2-4aea-8f3d-300f80942777" (UID: "b3868b2e-81d2-4aea-8f3d-300f80942777"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:36.506791 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506692    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-home" (OuterVolumeSpecName: "home") pod "b3868b2e-81d2-4aea-8f3d-300f80942777" (UID: "b3868b2e-81d2-4aea-8f3d-300f80942777"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:36.506954 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506864    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:36.506954 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506884    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:36.506954 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.506896    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:36.508749 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.508721    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-dshm" (OuterVolumeSpecName: "dshm") pod "b3868b2e-81d2-4aea-8f3d-300f80942777" (UID: "b3868b2e-81d2-4aea-8f3d-300f80942777"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:36.508872 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.508760    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3868b2e-81d2-4aea-8f3d-300f80942777-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b3868b2e-81d2-4aea-8f3d-300f80942777" (UID: "b3868b2e-81d2-4aea-8f3d-300f80942777"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:14:36.508872 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.508771    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3868b2e-81d2-4aea-8f3d-300f80942777-kube-api-access-gbbrf" (OuterVolumeSpecName: "kube-api-access-gbbrf") pod "b3868b2e-81d2-4aea-8f3d-300f80942777" (UID: "b3868b2e-81d2-4aea-8f3d-300f80942777"). InnerVolumeSpecName "kube-api-access-gbbrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:14:36.608312 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.608277    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbbrf\" (UniqueName: \"kubernetes.io/projected/b3868b2e-81d2-4aea-8f3d-300f80942777-kube-api-access-gbbrf\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:36.608312 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.608305    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3868b2e-81d2-4aea-8f3d-300f80942777-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:36.608312 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:36.608315    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3868b2e-81d2-4aea-8f3d-300f80942777-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:37.109131 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.109071    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4" podUID="32298aab-a27a-4176-9c57-68b3f6c0fd41" containerName="main" containerID="cri-o://3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca" gracePeriod=45
Jul 24 15:14:37.171243 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.171213    2581 generic.go:356] "Generic (PLEG): container finished" podID="b3868b2e-81d2-4aea-8f3d-300f80942777" containerID="d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337" exitCode=0
Jul 24 15:14:37.171408 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.171256    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"
Jul 24 15:14:37.171408 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.171264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5" event={"ID":"b3868b2e-81d2-4aea-8f3d-300f80942777","Type":"ContainerDied","Data":"d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337"}
Jul 24 15:14:37.171408 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.171295    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5" event={"ID":"b3868b2e-81d2-4aea-8f3d-300f80942777","Type":"ContainerDied","Data":"f4b7911ada1418d978e9fac26f3687662f506bd6d0f7e80dadf6a9dd93422cdf"}
Jul 24 15:14:37.171408 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.171314    2581 scope.go:117] "RemoveContainer" containerID="d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337"
Jul 24 15:14:37.192008 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.191979    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"]
Jul 24 15:14:37.198954 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.198920    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/auth-test-model-dc7d81-kserve-54b7df776c-zb9w5"]
Jul 24 15:14:37.219808 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.219650    2581 scope.go:117] "RemoveContainer" containerID="d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337"
Jul 24 15:14:37.220102 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:14:37.220023    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337\": container with ID starting with d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337 not found: ID does not exist" containerID="d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337"
Jul 24 15:14:37.220102 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.220060    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337"} err="failed to get container status \"d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337\": rpc error: code = NotFound desc = could not find container \"d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337\": container with ID starting with d1b75bc992ea06a1019e46d60f21ba77a4605f58fb0af46f290e9c2c14c30337 not found: ID does not exist"
Jul 24 15:14:37.336270 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.336241    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3868b2e-81d2-4aea-8f3d-300f80942777" path="/var/lib/kubelet/pods/b3868b2e-81d2-4aea-8f3d-300f80942777/volumes"
Jul 24 15:14:37.359766 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.359711    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:14:37.415950 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.415918    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6ttk\" (UniqueName: \"kubernetes.io/projected/32298aab-a27a-4176-9c57-68b3f6c0fd41-kube-api-access-v6ttk\") pod \"32298aab-a27a-4176-9c57-68b3f6c0fd41\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") "
Jul 24 15:14:37.416166 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.416009    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32298aab-a27a-4176-9c57-68b3f6c0fd41-tls-certs\") pod \"32298aab-a27a-4176-9c57-68b3f6c0fd41\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") "
Jul 24 15:14:37.416242 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.416173    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-home\") pod \"32298aab-a27a-4176-9c57-68b3f6c0fd41\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") "
Jul 24 15:14:37.416242 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.416198    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-model-cache\") pod \"32298aab-a27a-4176-9c57-68b3f6c0fd41\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") "
Jul 24 15:14:37.416242 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.416226    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-tmp-dir\") pod \"32298aab-a27a-4176-9c57-68b3f6c0fd41\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") "
Jul 24 15:14:37.416403 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.416250    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-dshm\") pod \"32298aab-a27a-4176-9c57-68b3f6c0fd41\" (UID: \"32298aab-a27a-4176-9c57-68b3f6c0fd41\") "
Jul 24 15:14:37.416449 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.416417    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "32298aab-a27a-4176-9c57-68b3f6c0fd41" (UID: "32298aab-a27a-4176-9c57-68b3f6c0fd41"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:37.416449 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.416430    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-model-cache" (OuterVolumeSpecName: "model-cache") pod "32298aab-a27a-4176-9c57-68b3f6c0fd41" (UID: "32298aab-a27a-4176-9c57-68b3f6c0fd41"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:37.416518 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.416447    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-home" (OuterVolumeSpecName: "home") pod "32298aab-a27a-4176-9c57-68b3f6c0fd41" (UID: "32298aab-a27a-4176-9c57-68b3f6c0fd41"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:37.416627 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.416608    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:37.416674 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.416632    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:37.416674 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.416647    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:37.418232 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.418211    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32298aab-a27a-4176-9c57-68b3f6c0fd41-kube-api-access-v6ttk" (OuterVolumeSpecName: "kube-api-access-v6ttk") pod "32298aab-a27a-4176-9c57-68b3f6c0fd41" (UID: "32298aab-a27a-4176-9c57-68b3f6c0fd41"). InnerVolumeSpecName "kube-api-access-v6ttk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:14:37.418317 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.418230    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-dshm" (OuterVolumeSpecName: "dshm") pod "32298aab-a27a-4176-9c57-68b3f6c0fd41" (UID: "32298aab-a27a-4176-9c57-68b3f6c0fd41"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:37.418395 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.418377    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32298aab-a27a-4176-9c57-68b3f6c0fd41-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "32298aab-a27a-4176-9c57-68b3f6c0fd41" (UID: "32298aab-a27a-4176-9c57-68b3f6c0fd41"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:14:37.517715 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.517678    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32298aab-a27a-4176-9c57-68b3f6c0fd41-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:37.517715 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.517708    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6ttk\" (UniqueName: \"kubernetes.io/projected/32298aab-a27a-4176-9c57-68b3f6c0fd41-kube-api-access-v6ttk\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:37.517715 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:37.517722    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32298aab-a27a-4176-9c57-68b3f6c0fd41-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:14:38.184439 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.184401    2581 generic.go:356] "Generic (PLEG): container finished" podID="32298aab-a27a-4176-9c57-68b3f6c0fd41" containerID="3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca" exitCode=0
Jul 24 15:14:38.184590 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.184447    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"
Jul 24 15:14:38.184590 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.184479    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4" event={"ID":"32298aab-a27a-4176-9c57-68b3f6c0fd41","Type":"ContainerDied","Data":"3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca"}
Jul 24 15:14:38.184590 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.184520    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4" event={"ID":"32298aab-a27a-4176-9c57-68b3f6c0fd41","Type":"ContainerDied","Data":"e8d552608fda1611c60764f6d6a9af64af08aa9972d8437faa1a6a39f816a35a"}
Jul 24 15:14:38.184590 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.184537    2581 scope.go:117] "RemoveContainer" containerID="3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca"
Jul 24 15:14:38.193844 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.193829    2581 scope.go:117] "RemoveContainer" containerID="3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca"
Jul 24 15:14:38.194047 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:14:38.194031    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca\": container with ID starting with 3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca not found: ID does not exist" containerID="3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca"
Jul 24 15:14:38.194086 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.194053    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca"} err="failed to get container status \"3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca\": rpc error: code = NotFound desc = could not find container \"3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca\": container with ID starting with 3270530711b19358fda972a8bcbbe26ac2db9aa3cc9d6f426df16dfcfd8a05ca not found: ID does not exist"
Jul 24 15:14:38.207036 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.207010    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"]
Jul 24 15:14:38.209747 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.209727    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/auth-test-model-8ea489-kserve-6946f4477f-6wdb4"]
Jul 24 15:14:38.453808 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.453722    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:38.454172 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.453804    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:38.466082 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:38.466057    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:39.201745 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:39.201719    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:14:39.335000 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:39.334958    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32298aab-a27a-4176-9c57-68b3f6c0fd41" path="/var/lib/kubelet/pods/32298aab-a27a-4176-9c57-68b3f6c0fd41/volumes"
Jul 24 15:14:51.894822 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.894786    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"]
Jul 24 15:14:51.895363 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.895347    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3868b2e-81d2-4aea-8f3d-300f80942777" containerName="main"
Jul 24 15:14:51.895417 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.895368    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3868b2e-81d2-4aea-8f3d-300f80942777" containerName="main"
Jul 24 15:14:51.895417 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.895387    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32298aab-a27a-4176-9c57-68b3f6c0fd41" containerName="main"
Jul 24 15:14:51.895417 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.895396    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="32298aab-a27a-4176-9c57-68b3f6c0fd41" containerName="main"
Jul 24 15:14:51.895521 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.895472    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3868b2e-81d2-4aea-8f3d-300f80942777" containerName="main"
Jul 24 15:14:51.895521 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.895484    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="32298aab-a27a-4176-9c57-68b3f6c0fd41" containerName="main"
Jul 24 15:14:51.898940 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.898916    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:51.903682 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.903660    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"kube-root-ca.crt\""
Jul 24 15:14:51.904529 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.904507    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"openshift-service-ca.crt\""
Jul 24 15:14:51.904774 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.904748    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"sub-test-model-8ea489-kserve-self-signed-certs\""
Jul 24 15:14:51.905927 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.905547    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"default-dockercfg-5c594\""
Jul 24 15:14:51.932426 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:51.932398    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"]
Jul 24 15:14:52.040662 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.040624    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:14:52.047677 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.047644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-dshm\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.047677 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.047679    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-home\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.047876 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.047708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-tmp-dir\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.047876 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.047781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-model-cache\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.047876 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.047817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa01934e-1033-45e4-8ba4-a3379765d69b-tls-certs\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.047978 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.047933    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9snr\" (UniqueName: \"kubernetes.io/projected/aa01934e-1033-45e4-8ba4-a3379765d69b-kube-api-access-x9snr\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.149081 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.148990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-model-cache\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.149081 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.149030    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa01934e-1033-45e4-8ba4-a3379765d69b-tls-certs\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.149302 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.149147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9snr\" (UniqueName: \"kubernetes.io/projected/aa01934e-1033-45e4-8ba4-a3379765d69b-kube-api-access-x9snr\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.149302 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.149190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-dshm\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.149302 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.149214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-home\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.149302 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.149234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-tmp-dir\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.149501 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.149426    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-model-cache\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.149555 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.149537    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-tmp-dir\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.149607 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.149586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-home\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.151337 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.151320    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-dshm\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.151607 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.151592    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa01934e-1033-45e4-8ba4-a3379765d69b-tls-certs\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.174765 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.174733    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9snr\" (UniqueName: \"kubernetes.io/projected/aa01934e-1033-45e4-8ba4-a3379765d69b-kube-api-access-x9snr\") pod \"sub-test-model-8ea489-kserve-fb99c948f-h7mhf\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") " pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.208414 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.208379    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:14:52.361460 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:52.361430    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"]
Jul 24 15:14:52.364426 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:14:52.364400    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa01934e_1033_45e4_8ba4_a3379765d69b.slice/crio-f487a8256b0cae5d12f412e90c6b3784023d58d9e49ff90542278c954a8a8f7d WatchSource:0}: Error finding container f487a8256b0cae5d12f412e90c6b3784023d58d9e49ff90542278c954a8a8f7d: Status 404 returned error can't find the container with id f487a8256b0cae5d12f412e90c6b3784023d58d9e49ff90542278c954a8a8f7d
Jul 24 15:14:53.260662 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:53.260626    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf" event={"ID":"aa01934e-1033-45e4-8ba4-a3379765d69b","Type":"ContainerStarted","Data":"59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660"}
Jul 24 15:14:53.261046 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:53.260670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf" event={"ID":"aa01934e-1033-45e4-8ba4-a3379765d69b","Type":"ContainerStarted","Data":"f487a8256b0cae5d12f412e90c6b3784023d58d9e49ff90542278c954a8a8f7d"}
Jul 24 15:14:53.287279 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:53.287228    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf" podStartSLOduration=2.287211455 podStartE2EDuration="2.287211455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:14:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:14:53.285695378 +0000 UTC m=+2571.789413347" watchObservedRunningTime="2026-07-24 15:14:53.287211455 +0000 UTC m=+2571.790929413"
Jul 24 15:14:55.452354 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:55.452322    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:14:59.687466 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:59.687429    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:14:59.802881 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:14:59.802842    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:00.195452 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.195417    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4"]
Jul 24 15:15:00.199230 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.199208    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4"
Jul 24 15:15:00.210337 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.210315    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:15:00.212527 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.212506    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x"]
Jul 24 15:15:00.215873 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.215854    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5"]
Jul 24 15:15:00.215972 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.215963    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x"
Jul 24 15:15:00.219023 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.219006    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5"
Jul 24 15:15:00.219324 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.219309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4"
Jul 24 15:15:00.226058 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.226041    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x"
Jul 24 15:15:00.252471 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.252343    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5"]
Jul 24 15:15:00.255251 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.255180    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x"]
Jul 24 15:15:00.261281 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.261254    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5"
Jul 24 15:15:00.302413 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.302336    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4"]
Jul 24 15:15:00.400688 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.400656    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x"]
Jul 24 15:15:00.405540 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:15:00.405514    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb12d1f2f_ee6a_4d78_96c1_0f60bd19ba0f.slice/crio-ca400174a40f411a1b5ec45156a74b3706fbc3062f75864aa25792c25d62b23f WatchSource:0}: Error finding container ca400174a40f411a1b5ec45156a74b3706fbc3062f75864aa25792c25d62b23f: Status 404 returned error can't find the container with id ca400174a40f411a1b5ec45156a74b3706fbc3062f75864aa25792c25d62b23f
Jul 24 15:15:00.442184 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.442161    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5"]
Jul 24 15:15:00.444685 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:15:00.444661    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod892d9632_478f_4532_9f37_f6d7749fec8d.slice/crio-a13db02f55454139305ef53f631f6f09eb7941995e5b12787740edad4fd64f96 WatchSource:0}: Error finding container a13db02f55454139305ef53f631f6f09eb7941995e5b12787740edad4fd64f96: Status 404 returned error can't find the container with id a13db02f55454139305ef53f631f6f09eb7941995e5b12787740edad4fd64f96
Jul 24 15:15:00.583073 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:00.583047    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4"]
Jul 24 15:15:00.584568 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:15:00.584528    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9edd38ee_c750_401d_b646_f106c2cd5081.slice/crio-3efbaf8964d6e69cb9b18b3277a82e8f41bac75ecb5b7b395d237243d054fd42 WatchSource:0}: Error finding container 3efbaf8964d6e69cb9b18b3277a82e8f41bac75ecb5b7b395d237243d054fd42: Status 404 returned error can't find the container with id 3efbaf8964d6e69cb9b18b3277a82e8f41bac75ecb5b7b395d237243d054fd42
Jul 24 15:15:01.304260 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:01.304225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4" event={"ID":"9edd38ee-c750-401d-b646-f106c2cd5081","Type":"ContainerStarted","Data":"24a3b93264017f8be937a5773038813146ff7be510394973016fb51b0604d072"}
Jul 24 15:15:01.304714 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:01.304283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4" event={"ID":"9edd38ee-c750-401d-b646-f106c2cd5081","Type":"ContainerStarted","Data":"3efbaf8964d6e69cb9b18b3277a82e8f41bac75ecb5b7b395d237243d054fd42"}
Jul 24 15:15:01.305762 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:01.305736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x" event={"ID":"b12d1f2f-ee6a-4d78-96c1-0f60bd19ba0f","Type":"ContainerStarted","Data":"1b9381273e12277120a3c1b0056d220bbbae333af184fac3c72d0528a5b2f3b4"}
Jul 24 15:15:01.305894 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:01.305778    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x" event={"ID":"b12d1f2f-ee6a-4d78-96c1-0f60bd19ba0f","Type":"ContainerStarted","Data":"ca400174a40f411a1b5ec45156a74b3706fbc3062f75864aa25792c25d62b23f"}
Jul 24 15:15:01.307214 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:01.307192    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5" event={"ID":"892d9632-478f-4532-9f37-f6d7749fec8d","Type":"ContainerStarted","Data":"5b432bcd1420d430266977740f3ef9d5c4c3fb9ea9644792d0451dc2a796cebb"}
Jul 24 15:15:01.307306 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:01.307237    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5" event={"ID":"892d9632-478f-4532-9f37-f6d7749fec8d","Type":"ContainerStarted","Data":"a13db02f55454139305ef53f631f6f09eb7941995e5b12787740edad4fd64f96"}
Jul 24 15:15:01.343348 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:01.343286    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4" podStartSLOduration=1.343264994 podStartE2EDuration="1.343264994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:15:01.340545035 +0000 UTC m=+2579.844262995" watchObservedRunningTime="2026-07-24 15:15:01.343264994 +0000 UTC m=+2579.846982953"
Jul 24 15:15:01.366538 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:01.366490    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x" podStartSLOduration=1.366473617 podStartE2EDuration="1.366473617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:15:01.366114201 +0000 UTC m=+2579.869832158" watchObservedRunningTime="2026-07-24 15:15:01.366473617 +0000 UTC m=+2579.870191574"
Jul 24 15:15:01.408868 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:01.408815    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5" podStartSLOduration=1.408798245 podStartE2EDuration="1.408798245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:15:01.408070653 +0000 UTC m=+2579.911788622" watchObservedRunningTime="2026-07-24 15:15:01.408798245 +0000 UTC m=+2579.912516202"
Jul 24 15:15:02.208886 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:02.208842    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:15:02.208886 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:02.208888    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:15:02.221313 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:02.221282    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:15:02.313868 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:02.313835    2581 generic.go:356] "Generic (PLEG): container finished" podID="b12d1f2f-ee6a-4d78-96c1-0f60bd19ba0f" containerID="1b9381273e12277120a3c1b0056d220bbbae333af184fac3c72d0528a5b2f3b4" exitCode=0
Jul 24 15:15:02.314300 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:02.313921    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x" event={"ID":"b12d1f2f-ee6a-4d78-96c1-0f60bd19ba0f","Type":"ContainerDied","Data":"1b9381273e12277120a3c1b0056d220bbbae333af184fac3c72d0528a5b2f3b4"}
Jul 24 15:15:02.315435 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:02.315414    2581 generic.go:356] "Generic (PLEG): container finished" podID="892d9632-478f-4532-9f37-f6d7749fec8d" containerID="5b432bcd1420d430266977740f3ef9d5c4c3fb9ea9644792d0451dc2a796cebb" exitCode=0
Jul 24 15:15:02.315513 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:02.315481    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5" event={"ID":"892d9632-478f-4532-9f37-f6d7749fec8d","Type":"ContainerDied","Data":"5b432bcd1420d430266977740f3ef9d5c4c3fb9ea9644792d0451dc2a796cebb"}
Jul 24 15:15:02.316846 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:02.316827    2581 generic.go:356] "Generic (PLEG): container finished" podID="9edd38ee-c750-401d-b646-f106c2cd5081" containerID="24a3b93264017f8be937a5773038813146ff7be510394973016fb51b0604d072" exitCode=0
Jul 24 15:15:02.316947 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:02.316904    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4" event={"ID":"9edd38ee-c750-401d-b646-f106c2cd5081","Type":"ContainerDied","Data":"24a3b93264017f8be937a5773038813146ff7be510394973016fb51b0604d072"}
Jul 24 15:15:02.328164 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:02.328145    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:15:03.469900 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:03.469879    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x"
Jul 24 15:15:03.501937 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:03.501905    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4"
Jul 24 15:15:03.505838 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:03.505816    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5"
Jul 24 15:15:04.327581 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:04.327555    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5"
Jul 24 15:15:04.327581 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:04.327571    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5" event={"ID":"892d9632-478f-4532-9f37-f6d7749fec8d","Type":"ContainerDied","Data":"a13db02f55454139305ef53f631f6f09eb7941995e5b12787740edad4fd64f96"}
Jul 24 15:15:04.327829 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:04.327620    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a13db02f55454139305ef53f631f6f09eb7941995e5b12787740edad4fd64f96"
Jul 24 15:15:04.328968 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:04.328949    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4"
Jul 24 15:15:04.329085 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:04.328993    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4" event={"ID":"9edd38ee-c750-401d-b646-f106c2cd5081","Type":"ContainerDied","Data":"3efbaf8964d6e69cb9b18b3277a82e8f41bac75ecb5b7b395d237243d054fd42"}
Jul 24 15:15:04.329085 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:04.329032    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3efbaf8964d6e69cb9b18b3277a82e8f41bac75ecb5b7b395d237243d054fd42"
Jul 24 15:15:04.330341 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:04.330326    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x"
Jul 24 15:15:04.330455 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:04.330338    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-hdn7x" event={"ID":"b12d1f2f-ee6a-4d78-96c1-0f60bd19ba0f","Type":"ContainerDied","Data":"ca400174a40f411a1b5ec45156a74b3706fbc3062f75864aa25792c25d62b23f"}
Jul 24 15:15:04.330455 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:04.330376    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca400174a40f411a1b5ec45156a74b3706fbc3062f75864aa25792c25d62b23f"
Jul 24 15:15:09.789632 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:09.789599    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:15:26.108888 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:26.108813    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:15:32.731531 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:32.731488    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"]
Jul 24 15:15:32.759874 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:32.759843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:33.577669 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:33.577633    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"]
Jul 24 15:15:33.596918 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:33.596885    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:15:44.725233 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:44.725196    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:15:47.752298 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:47.752233    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf" podUID="9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" containerName="main" containerID="cri-o://913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b" gracePeriod=45
Jul 24 15:15:47.998611 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:47.998582    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:15:48.164134 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.164082    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tls-certs\") pod \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") "
Jul 24 15:15:48.164364 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.164140    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvb8s\" (UniqueName: \"kubernetes.io/projected/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-kube-api-access-pvb8s\") pod \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") "
Jul 24 15:15:48.164364 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.164182    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tmp-dir\") pod \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") "
Jul 24 15:15:48.164364 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.164205    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-dshm\") pod \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") "
Jul 24 15:15:48.164364 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.164264    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-home\") pod \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") "
Jul 24 15:15:48.164364 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.164288    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-model-cache\") pod \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\" (UID: \"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1\") "
Jul 24 15:15:48.164627 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.164487    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" (UID: "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:48.164627 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.164502    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-home" (OuterVolumeSpecName: "home") pod "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" (UID: "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:48.164730 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.164620    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-model-cache" (OuterVolumeSpecName: "model-cache") pod "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" (UID: "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:48.164730 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.164631    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:48.164730 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.164669    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:48.166379 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.166356    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-dshm" (OuterVolumeSpecName: "dshm") pod "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" (UID: "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:48.166478 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.166414    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-kube-api-access-pvb8s" (OuterVolumeSpecName: "kube-api-access-pvb8s") pod "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" (UID: "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1"). InnerVolumeSpecName "kube-api-access-pvb8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:15:48.166478 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.166421    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" (UID: "9a5ec87c-b71f-48bd-ab87-06bb330fd3b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:15:48.265226 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.265196    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:48.265226 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.265218    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:48.265226 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.265228    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvb8s\" (UniqueName: \"kubernetes.io/projected/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-kube-api-access-pvb8s\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:48.265457 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.265237    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:48.543524 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.543440    2581 generic.go:356] "Generic (PLEG): container finished" podID="9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" containerID="913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b" exitCode=0
Jul 24 15:15:48.543524 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.543476    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"
Jul 24 15:15:48.543707 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.543527    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf" event={"ID":"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1","Type":"ContainerDied","Data":"913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b"}
Jul 24 15:15:48.543707 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.543565    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf" event={"ID":"9a5ec87c-b71f-48bd-ab87-06bb330fd3b1","Type":"ContainerDied","Data":"448c7d0ba0cb5a4a9217a4034d8d077af44008116c52344818c64509cb738bb8"}
Jul 24 15:15:48.543707 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.543583    2581 scope.go:117] "RemoveContainer" containerID="913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b"
Jul 24 15:15:48.553072 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.553054    2581 scope.go:117] "RemoveContainer" containerID="913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b"
Jul 24 15:15:48.553378 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:15:48.553359    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b\": container with ID starting with 913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b not found: ID does not exist" containerID="913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b"
Jul 24 15:15:48.553438 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.553388    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b"} err="failed to get container status \"913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b\": rpc error: code = NotFound desc = could not find container \"913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b\": container with ID starting with 913460450ba0afcaeba261a2919d291a1bd44b13ac97e62e71068ea620fea12b not found: ID does not exist"
Jul 24 15:15:48.567516 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.567486    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"]
Jul 24 15:15:48.570087 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.570060    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/sub-test-model-dc7d81-kserve-7c446d7689-skpwf"]
Jul 24 15:15:48.596000 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.595964    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf" podUID="aa01934e-1033-45e4-8ba4-a3379765d69b" containerName="main" containerID="cri-o://59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660" gracePeriod=45
Jul 24 15:15:48.845948 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.845924    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:15:48.971459 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.971418    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9snr\" (UniqueName: \"kubernetes.io/projected/aa01934e-1033-45e4-8ba4-a3379765d69b-kube-api-access-x9snr\") pod \"aa01934e-1033-45e4-8ba4-a3379765d69b\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") "
Jul 24 15:15:48.971459 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.971464    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-tmp-dir\") pod \"aa01934e-1033-45e4-8ba4-a3379765d69b\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") "
Jul 24 15:15:48.971714 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.971490    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-model-cache\") pod \"aa01934e-1033-45e4-8ba4-a3379765d69b\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") "
Jul 24 15:15:48.971714 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.971582    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-dshm\") pod \"aa01934e-1033-45e4-8ba4-a3379765d69b\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") "
Jul 24 15:15:48.971714 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.971613    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-home\") pod \"aa01934e-1033-45e4-8ba4-a3379765d69b\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") "
Jul 24 15:15:48.971714 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.971637    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa01934e-1033-45e4-8ba4-a3379765d69b-tls-certs\") pod \"aa01934e-1033-45e4-8ba4-a3379765d69b\" (UID: \"aa01934e-1033-45e4-8ba4-a3379765d69b\") "
Jul 24 15:15:48.971900 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.971757    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aa01934e-1033-45e4-8ba4-a3379765d69b" (UID: "aa01934e-1033-45e4-8ba4-a3379765d69b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:48.971900 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.971819    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-model-cache" (OuterVolumeSpecName: "model-cache") pod "aa01934e-1033-45e4-8ba4-a3379765d69b" (UID: "aa01934e-1033-45e4-8ba4-a3379765d69b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:48.971900 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.971858    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-home" (OuterVolumeSpecName: "home") pod "aa01934e-1033-45e4-8ba4-a3379765d69b" (UID: "aa01934e-1033-45e4-8ba4-a3379765d69b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:48.972022 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.972002    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:48.972022 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.972019    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:48.972088 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.972031    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:48.973625 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.973600    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa01934e-1033-45e4-8ba4-a3379765d69b-kube-api-access-x9snr" (OuterVolumeSpecName: "kube-api-access-x9snr") pod "aa01934e-1033-45e4-8ba4-a3379765d69b" (UID: "aa01934e-1033-45e4-8ba4-a3379765d69b"). InnerVolumeSpecName "kube-api-access-x9snr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:15:48.974037 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.974013    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa01934e-1033-45e4-8ba4-a3379765d69b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aa01934e-1033-45e4-8ba4-a3379765d69b" (UID: "aa01934e-1033-45e4-8ba4-a3379765d69b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:15:48.974037 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:48.974024    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-dshm" (OuterVolumeSpecName: "dshm") pod "aa01934e-1033-45e4-8ba4-a3379765d69b" (UID: "aa01934e-1033-45e4-8ba4-a3379765d69b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:49.072747 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.072713    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9snr\" (UniqueName: \"kubernetes.io/projected/aa01934e-1033-45e4-8ba4-a3379765d69b-kube-api-access-x9snr\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:49.072747 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.072740    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa01934e-1033-45e4-8ba4-a3379765d69b-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:49.072747 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.072752    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa01934e-1033-45e4-8ba4-a3379765d69b-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:15:49.334457 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.334422    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" path="/var/lib/kubelet/pods/9a5ec87c-b71f-48bd-ab87-06bb330fd3b1/volumes"
Jul 24 15:15:49.550457 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.550424    2581 generic.go:356] "Generic (PLEG): container finished" podID="aa01934e-1033-45e4-8ba4-a3379765d69b" containerID="59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660" exitCode=0
Jul 24 15:15:49.550636 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.550470    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"
Jul 24 15:15:49.550636 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.550499    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf" event={"ID":"aa01934e-1033-45e4-8ba4-a3379765d69b","Type":"ContainerDied","Data":"59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660"}
Jul 24 15:15:49.550636 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.550541    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf" event={"ID":"aa01934e-1033-45e4-8ba4-a3379765d69b","Type":"ContainerDied","Data":"f487a8256b0cae5d12f412e90c6b3784023d58d9e49ff90542278c954a8a8f7d"}
Jul 24 15:15:49.550636 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.550565    2581 scope.go:117] "RemoveContainer" containerID="59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660"
Jul 24 15:15:49.559142 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.559127    2581 scope.go:117] "RemoveContainer" containerID="59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660"
Jul 24 15:15:49.559411 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:15:49.559391    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660\": container with ID starting with 59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660 not found: ID does not exist" containerID="59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660"
Jul 24 15:15:49.559458 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.559421    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660"} err="failed to get container status \"59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660\": rpc error: code = NotFound desc = could not find container \"59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660\": container with ID starting with 59980307e3ad494ec047346afe5d452e82e3f62496c82e5d90469beda1bf4660 not found: ID does not exist"
Jul 24 15:15:49.571475 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.571449    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"]
Jul 24 15:15:49.580119 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:49.580082    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/sub-test-model-8ea489-kserve-fb99c948f-h7mhf"]
Jul 24 15:15:51.333748 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:15:51.333713    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa01934e-1033-45e4-8ba4-a3379765d69b" path="/var/lib/kubelet/pods/aa01934e-1033-45e4-8ba4-a3379765d69b/volumes"
Jul 24 15:16:02.826448 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826418    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"]
Jul 24 15:16:02.826839 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826778    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa01934e-1033-45e4-8ba4-a3379765d69b" containerName="main"
Jul 24 15:16:02.826839 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826789    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa01934e-1033-45e4-8ba4-a3379765d69b" containerName="main"
Jul 24 15:16:02.826839 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826798    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b12d1f2f-ee6a-4d78-96c1-0f60bd19ba0f" containerName="cleanup"
Jul 24 15:16:02.826839 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826803    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b12d1f2f-ee6a-4d78-96c1-0f60bd19ba0f" containerName="cleanup"
Jul 24 15:16:02.826839 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826815    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" containerName="main"
Jul 24 15:16:02.826839 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826822    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" containerName="main"
Jul 24 15:16:02.826839 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826829    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="892d9632-478f-4532-9f37-f6d7749fec8d" containerName="cleanup"
Jul 24 15:16:02.826839 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826833    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="892d9632-478f-4532-9f37-f6d7749fec8d" containerName="cleanup"
Jul 24 15:16:02.826839 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826841    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9edd38ee-c750-401d-b646-f106c2cd5081" containerName="cleanup"
Jul 24 15:16:02.827140 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826848    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9edd38ee-c750-401d-b646-f106c2cd5081" containerName="cleanup"
Jul 24 15:16:02.827140 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826907    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b12d1f2f-ee6a-4d78-96c1-0f60bd19ba0f" containerName="cleanup"
Jul 24 15:16:02.827140 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826914    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9edd38ee-c750-401d-b646-f106c2cd5081" containerName="cleanup"
Jul 24 15:16:02.827140 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826926    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a5ec87c-b71f-48bd-ab87-06bb330fd3b1" containerName="main"
Jul 24 15:16:02.827140 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826933    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa01934e-1033-45e4-8ba4-a3379765d69b" containerName="main"
Jul 24 15:16:02.827140 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.826939    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="892d9632-478f-4532-9f37-f6d7749fec8d" containerName="cleanup"
Jul 24 15:16:02.831494 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.831477    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.834363 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.834341    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-dc7d81\"/\"kube-root-ca.crt\""
Jul 24 15:16:02.835508 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.835483    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-dc7d81\"/\"default-dockercfg-vtxpb\""
Jul 24 15:16:02.835508 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.835496    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-dc7d81\"/\"openshift-service-ca.crt\""
Jul 24 15:16:02.835644 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.835590    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-dc7d81\"/\"rate-test-model-dc7d81-kserve-self-signed-certs\""
Jul 24 15:16:02.846558 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.846537    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"]
Jul 24 15:16:02.861761 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.861733    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:16:02.892361 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.892332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-home\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.892497 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.892370    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-tmp-dir\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.892497 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.892398    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-model-cache\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.892497 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.892439    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b30b3c6-1d55-4883-a085-30ece79b4384-tls-certs\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.892497 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.892483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-dshm\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.892653 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.892545    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flzxb\" (UniqueName: \"kubernetes.io/projected/5b30b3c6-1d55-4883-a085-30ece79b4384-kube-api-access-flzxb\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.993472 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.993439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flzxb\" (UniqueName: \"kubernetes.io/projected/5b30b3c6-1d55-4883-a085-30ece79b4384-kube-api-access-flzxb\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.993472 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.993476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-home\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.993706 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.993497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-tmp-dir\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.993706 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.993523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-model-cache\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.993706 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.993555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b30b3c6-1d55-4883-a085-30ece79b4384-tls-certs\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.993706 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.993602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-dshm\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.993953 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.993931    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-home\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.993996 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.993972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-tmp-dir\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.994030 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.993981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-model-cache\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.996024 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.996001    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-dshm\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:02.996344 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:02.996322    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b30b3c6-1d55-4883-a085-30ece79b4384-tls-certs\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:03.002286 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:03.002265    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flzxb\" (UniqueName: \"kubernetes.io/projected/5b30b3c6-1d55-4883-a085-30ece79b4384-kube-api-access-flzxb\") pod \"rate-test-model-dc7d81-kserve-5fffbc587-d9g7n\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") " pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:03.140362 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:03.140329    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:03.280642 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:03.280613    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"]
Jul 24 15:16:03.282719 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:16:03.282689    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b30b3c6_1d55_4883_a085_30ece79b4384.slice/crio-3340e91da11300401f9a5bf2ac0f3fed13a255bb983536f1118b7cc02d92d6ac WatchSource:0}: Error finding container 3340e91da11300401f9a5bf2ac0f3fed13a255bb983536f1118b7cc02d92d6ac: Status 404 returned error can't find the container with id 3340e91da11300401f9a5bf2ac0f3fed13a255bb983536f1118b7cc02d92d6ac
Jul 24 15:16:03.595124 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:03.595034    2581 scope.go:117] "RemoveContainer" containerID="6eade326bca8b124484fe4e0e05ed56b007342e151a6052f587a254ad3ad8c51"
Jul 24 15:16:03.602923 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:03.602899    2581 scope.go:117] "RemoveContainer" containerID="d67422930f3ab43b5edd4bf5f8318f32691f1ab9d301cca326f7a0b01596e424"
Jul 24 15:16:03.622151 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:03.622122    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n" event={"ID":"5b30b3c6-1d55-4883-a085-30ece79b4384","Type":"ContainerStarted","Data":"1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e"}
Jul 24 15:16:03.622245 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:03.622167    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n" event={"ID":"5b30b3c6-1d55-4883-a085-30ece79b4384","Type":"ContainerStarted","Data":"3340e91da11300401f9a5bf2ac0f3fed13a255bb983536f1118b7cc02d92d6ac"}
Jul 24 15:16:03.649548 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:03.649496    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n" podStartSLOduration=1.649482831 podStartE2EDuration="1.649482831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:16:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:16:03.64812631 +0000 UTC m=+2642.151844265" watchObservedRunningTime="2026-07-24 15:16:03.649482831 +0000 UTC m=+2642.153200831"
Jul 24 15:16:06.127276 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:06.127245    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:16:13.140630 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:13.140585    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:13.140630 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:13.140632    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:13.152891 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:13.152867    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:13.683132 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:13.683102    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:16:21.224323 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.224289    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"]
Jul 24 15:16:21.227719 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.227701    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.231372 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.231341    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"kube-root-ca.crt\""
Jul 24 15:16:21.231372 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.231359    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"default-dockercfg-5c594\""
Jul 24 15:16:21.231561 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.231373    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"openshift-service-ca.crt\""
Jul 24 15:16:21.231692 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.231676    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8ea489\"/\"rate-test-model-8ea489-kserve-self-signed-certs\""
Jul 24 15:16:21.237902 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.237881    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"]
Jul 24 15:16:21.270458 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.270428    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:16:21.363231 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.363198    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-dshm\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.363231 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.363236    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hqpn\" (UniqueName: \"kubernetes.io/projected/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-kube-api-access-5hqpn\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.363453 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.363256    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tls-certs\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.363453 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.363290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tmp-dir\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.363453 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.363334    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-model-cache\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.363453 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.363412    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-home\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.464478 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.464446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-model-cache\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.464681 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.464512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-home\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.464681 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.464550    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-dshm\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.464681 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.464566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hqpn\" (UniqueName: \"kubernetes.io/projected/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-kube-api-access-5hqpn\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.464681 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.464586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tls-certs\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.464681 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.464609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tmp-dir\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.464950 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.464848    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-model-cache\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.464950 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.464888    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-home\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.465207 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.465185    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tmp-dir\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.466851 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.466828    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-dshm\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.467144 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.467125    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tls-certs\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.476328 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.476274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hqpn\" (UniqueName: \"kubernetes.io/projected/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-kube-api-access-5hqpn\") pod \"rate-test-model-8ea489-kserve-5c7c654b55-dhqqq\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") " pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.537412 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.537377    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:21.671992 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.671970    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"]
Jul 24 15:16:21.673890 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:16:21.673865    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbe4aac8_424a_4242_bf6e_0dccbf2af7ff.slice/crio-62fb1acd5fcc4fc7a2e47c2fd6ac678e4bd96169f9bc2000efac9d7962c78eb9 WatchSource:0}: Error finding container 62fb1acd5fcc4fc7a2e47c2fd6ac678e4bd96169f9bc2000efac9d7962c78eb9: Status 404 returned error can't find the container with id 62fb1acd5fcc4fc7a2e47c2fd6ac678e4bd96169f9bc2000efac9d7962c78eb9
Jul 24 15:16:21.715526 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:21.715489    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq" event={"ID":"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff","Type":"ContainerStarted","Data":"62fb1acd5fcc4fc7a2e47c2fd6ac678e4bd96169f9bc2000efac9d7962c78eb9"}
Jul 24 15:16:22.722278 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:22.722247    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq" event={"ID":"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff","Type":"ContainerStarted","Data":"292402792ab8f5ec967798283dbffeea82f28298d134de27469db6d4d4ccc45e"}
Jul 24 15:16:22.781879 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:22.781803    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq" podStartSLOduration=1.781782934 podStartE2EDuration="1.781782934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:16:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:16:22.779704738 +0000 UTC m=+2661.283422696" watchObservedRunningTime="2026-07-24 15:16:22.781782934 +0000 UTC m=+2661.285500893"
Jul 24 15:16:24.824227 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:24.824191    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:16:31.538285 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:31.538198    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:31.538854 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:31.538345    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:31.550770 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:31.550745    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:31.775260 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:31.775226    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:16:45.806307 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:16:45.806270    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:17:03.632435 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:03.632360    2581 scope.go:117] "RemoveContainer" containerID="66c67f428871598966d500bce73b14276c89e3306b7e979ea1c5f5d5b1dddaea"
Jul 24 15:17:03.640897 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:03.640878    2581 scope.go:117] "RemoveContainer" containerID="848a8874add967011bfa73d0f47dacae584057137a3f28ab902be58c35f0f18c"
Jul 24 15:17:06.345655 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:06.345622    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:17:13.975785 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:13.975752    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"]
Jul 24 15:17:14.000473 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:14.000422    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:14.801200 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:14.801168    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"]
Jul 24 15:17:14.826565 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:14.826530    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:15.100965 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:15.100879    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:15.763832 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:15.763793    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7459-140624082966704"]
Jul 24 15:17:15.766996 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:15.766973    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082966704"
Jul 24 15:17:15.770465 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:15.770446    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-z9gsk\""
Jul 24 15:17:15.782485 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:15.782459    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624082966704"]
Jul 24 15:17:15.876818 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:15.876781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snvvh\" (UniqueName: \"kubernetes.io/projected/0e1857bf-3bad-474e-a952-afe26255adf6-kube-api-access-snvvh\") pod \"test-curl-7459-140624082966704\" (UID: \"0e1857bf-3bad-474e-a952-afe26255adf6\") " pod="opendatahub/test-curl-7459-140624082966704"
Jul 24 15:17:15.977770 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:15.977738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snvvh\" (UniqueName: \"kubernetes.io/projected/0e1857bf-3bad-474e-a952-afe26255adf6-kube-api-access-snvvh\") pod \"test-curl-7459-140624082966704\" (UID: \"0e1857bf-3bad-474e-a952-afe26255adf6\") " pod="opendatahub/test-curl-7459-140624082966704"
Jul 24 15:17:15.988623 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:15.988598    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snvvh\" (UniqueName: \"kubernetes.io/projected/0e1857bf-3bad-474e-a952-afe26255adf6-kube-api-access-snvvh\") pod \"test-curl-7459-140624082966704\" (UID: \"0e1857bf-3bad-474e-a952-afe26255adf6\") " pod="opendatahub/test-curl-7459-140624082966704"
Jul 24 15:17:16.076641 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:16.076553    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082966704"
Jul 24 15:17:16.215353 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:16.215319    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624082966704"]
Jul 24 15:17:16.217905 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:17:16.217879    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e1857bf_3bad_474e_a952_afe26255adf6.slice/crio-843c10ba265edef7b9309051ac91f5a0fe4f69d000edbc054333556d25e5d3ea WatchSource:0}: Error finding container 843c10ba265edef7b9309051ac91f5a0fe4f69d000edbc054333556d25e5d3ea: Status 404 returned error can't find the container with id 843c10ba265edef7b9309051ac91f5a0fe4f69d000edbc054333556d25e5d3ea
Jul 24 15:17:16.219583 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:16.219567    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:17:16.978044 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:16.978009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624082966704" event={"ID":"0e1857bf-3bad-474e-a952-afe26255adf6","Type":"ContainerStarted","Data":"5e5c4e1feb9e9399d87c08a21844620562118a6bf445330e58db281bb9a92575"}
Jul 24 15:17:16.978221 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:16.978053    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624082966704" event={"ID":"0e1857bf-3bad-474e-a952-afe26255adf6","Type":"ContainerStarted","Data":"843c10ba265edef7b9309051ac91f5a0fe4f69d000edbc054333556d25e5d3ea"}
Jul 24 15:17:16.998310 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:16.998269    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7459-140624082966704" podStartSLOduration=1.317610936 podStartE2EDuration="1.998254328s" totalImagesPullingTime="680.643392ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:17:16.996210663 +0000 UTC m=+2715.499928618" watchObservedRunningTime="2026-07-24 15:17:16.998254328 +0000 UTC m=+2715.501972286"
Jul 24 15:17:17.117696 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:17.117662    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7459-140624082966704_0e1857bf-3bad-474e-a952-afe26255adf6/test-curl-7459-140624082966704/0.log"
Jul 24 15:17:17.984104 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:17.984059    2581 generic.go:356] "Generic (PLEG): container finished" podID="0e1857bf-3bad-474e-a952-afe26255adf6" containerID="5e5c4e1feb9e9399d87c08a21844620562118a6bf445330e58db281bb9a92575" exitCode=0
Jul 24 15:17:17.984504 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:17.984144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624082966704" event={"ID":"0e1857bf-3bad-474e-a952-afe26255adf6","Type":"ContainerDied","Data":"5e5c4e1feb9e9399d87c08a21844620562118a6bf445330e58db281bb9a92575"}
Jul 24 15:17:19.115830 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.115804    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082966704"
Jul 24 15:17:19.162248 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.162207    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7459-140624082966704"]
Jul 24 15:17:19.165232 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.165207    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7459-140624082966704"]
Jul 24 15:17:19.195400 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.195362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:19.206391 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.206373    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snvvh\" (UniqueName: \"kubernetes.io/projected/0e1857bf-3bad-474e-a952-afe26255adf6-kube-api-access-snvvh\") pod \"0e1857bf-3bad-474e-a952-afe26255adf6\" (UID: \"0e1857bf-3bad-474e-a952-afe26255adf6\") "
Jul 24 15:17:19.208504 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.208477    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1857bf-3bad-474e-a952-afe26255adf6-kube-api-access-snvvh" (OuterVolumeSpecName: "kube-api-access-snvvh") pod "0e1857bf-3bad-474e-a952-afe26255adf6" (UID: "0e1857bf-3bad-474e-a952-afe26255adf6"). InnerVolumeSpecName "kube-api-access-snvvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:17:19.266740 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.266658    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:19.285449 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.285420    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7459-140624082966832"]
Jul 24 15:17:19.285780 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.285767    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1857bf-3bad-474e-a952-afe26255adf6" containerName="test-curl-7459-140624082966704"
Jul 24 15:17:19.285840 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.285781    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1857bf-3bad-474e-a952-afe26255adf6" containerName="test-curl-7459-140624082966704"
Jul 24 15:17:19.285877 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.285852    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e1857bf-3bad-474e-a952-afe26255adf6" containerName="test-curl-7459-140624082966704"
Jul 24 15:17:19.288825 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.288809    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082966832"
Jul 24 15:17:19.295565 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.295543    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624082966832"]
Jul 24 15:17:19.307226 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.307197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rthbp\" (UniqueName: \"kubernetes.io/projected/b273463b-551b-486d-8882-715a2b83442a-kube-api-access-rthbp\") pod \"test-curl-7459-140624082966832\" (UID: \"b273463b-551b-486d-8882-715a2b83442a\") " pod="opendatahub/test-curl-7459-140624082966832"
Jul 24 15:17:19.307374 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.307308    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snvvh\" (UniqueName: \"kubernetes.io/projected/0e1857bf-3bad-474e-a952-afe26255adf6-kube-api-access-snvvh\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:19.334453 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.334423    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e1857bf-3bad-474e-a952-afe26255adf6" path="/var/lib/kubelet/pods/0e1857bf-3bad-474e-a952-afe26255adf6/volumes"
Jul 24 15:17:19.408638 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.408604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rthbp\" (UniqueName: \"kubernetes.io/projected/b273463b-551b-486d-8882-715a2b83442a-kube-api-access-rthbp\") pod \"test-curl-7459-140624082966832\" (UID: \"b273463b-551b-486d-8882-715a2b83442a\") " pod="opendatahub/test-curl-7459-140624082966832"
Jul 24 15:17:19.417763 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.417732    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rthbp\" (UniqueName: \"kubernetes.io/projected/b273463b-551b-486d-8882-715a2b83442a-kube-api-access-rthbp\") pod \"test-curl-7459-140624082966832\" (UID: \"b273463b-551b-486d-8882-715a2b83442a\") " pod="opendatahub/test-curl-7459-140624082966832"
Jul 24 15:17:19.598568 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.598470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082966832"
Jul 24 15:17:19.724756 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.724733    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624082966832"]
Jul 24 15:17:19.726397 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:17:19.726376    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb273463b_551b_486d_8882_715a2b83442a.slice/crio-8a3723f61edf50006569f3f0fd06906e7c38e0c04e0d883e5495dc8baa826f81 WatchSource:0}: Error finding container 8a3723f61edf50006569f3f0fd06906e7c38e0c04e0d883e5495dc8baa826f81: Status 404 returned error can't find the container with id 8a3723f61edf50006569f3f0fd06906e7c38e0c04e0d883e5495dc8baa826f81
Jul 24 15:17:19.994443 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.994406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624082966832" event={"ID":"b273463b-551b-486d-8882-715a2b83442a","Type":"ContainerStarted","Data":"8a3723f61edf50006569f3f0fd06906e7c38e0c04e0d883e5495dc8baa826f81"}
Jul 24 15:17:19.995607 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.995584    2581 scope.go:117] "RemoveContainer" containerID="5e5c4e1feb9e9399d87c08a21844620562118a6bf445330e58db281bb9a92575"
Jul 24 15:17:19.995607 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:19.995599    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082966704"
Jul 24 15:17:20.504746 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:20.504709    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:17:21.002522 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:21.002489    2581 generic.go:356] "Generic (PLEG): container finished" podID="b273463b-551b-486d-8882-715a2b83442a" containerID="ad14de9e3f61454a0610f377437833c427274afd62d96d08ed8705102994c091" exitCode=0
Jul 24 15:17:21.002703 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:21.002536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624082966832" event={"ID":"b273463b-551b-486d-8882-715a2b83442a","Type":"ContainerDied","Data":"ad14de9e3f61454a0610f377437833c427274afd62d96d08ed8705102994c091"}
Jul 24 15:17:22.141560 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.141537    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082966832"
Jul 24 15:17:22.166429 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.162445    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7459-140624082966832_b273463b-551b-486d-8882-715a2b83442a/test-curl-7459-140624082966832/0.log"
Jul 24 15:17:22.185069 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.185039    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7459-140624082966832"]
Jul 24 15:17:22.190031 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.190009    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7459-140624082966832"]
Jul 24 15:17:22.232441 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.232417    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rthbp\" (UniqueName: \"kubernetes.io/projected/b273463b-551b-486d-8882-715a2b83442a-kube-api-access-rthbp\") pod \"b273463b-551b-486d-8882-715a2b83442a\" (UID: \"b273463b-551b-486d-8882-715a2b83442a\") "
Jul 24 15:17:22.234524 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.234502    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b273463b-551b-486d-8882-715a2b83442a-kube-api-access-rthbp" (OuterVolumeSpecName: "kube-api-access-rthbp") pod "b273463b-551b-486d-8882-715a2b83442a" (UID: "b273463b-551b-486d-8882-715a2b83442a"). InnerVolumeSpecName "kube-api-access-rthbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:17:22.334137 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.334042    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rthbp\" (UniqueName: \"kubernetes.io/projected/b273463b-551b-486d-8882-715a2b83442a-kube-api-access-rthbp\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:22.347489 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.347456    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7459-140624088332016"]
Jul 24 15:17:22.347851 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.347836    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b273463b-551b-486d-8882-715a2b83442a" containerName="test-curl-7459-140624082966832"
Jul 24 15:17:22.347851 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.347852    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b273463b-551b-486d-8882-715a2b83442a" containerName="test-curl-7459-140624082966832"
Jul 24 15:17:22.347952 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.347917    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b273463b-551b-486d-8882-715a2b83442a" containerName="test-curl-7459-140624082966832"
Jul 24 15:17:22.350821 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.350807    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:22.358022 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.357883    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624088332016"]
Jul 24 15:17:22.435248 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.435213    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqqt2\" (UniqueName: \"kubernetes.io/projected/904e997b-28fd-43bb-a9bc-b4d3f6940040-kube-api-access-pqqt2\") pod \"test-curl-7459-140624088332016\" (UID: \"904e997b-28fd-43bb-a9bc-b4d3f6940040\") " pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:22.535965 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.535931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqt2\" (UniqueName: \"kubernetes.io/projected/904e997b-28fd-43bb-a9bc-b4d3f6940040-kube-api-access-pqqt2\") pod \"test-curl-7459-140624088332016\" (UID: \"904e997b-28fd-43bb-a9bc-b4d3f6940040\") " pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:22.545359 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.545327    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqqt2\" (UniqueName: \"kubernetes.io/projected/904e997b-28fd-43bb-a9bc-b4d3f6940040-kube-api-access-pqqt2\") pod \"test-curl-7459-140624088332016\" (UID: \"904e997b-28fd-43bb-a9bc-b4d3f6940040\") " pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:22.660932 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.660906    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:22.788511 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:22.788488    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624088332016"]
Jul 24 15:17:22.790684 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:17:22.790660    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod904e997b_28fd_43bb_a9bc_b4d3f6940040.slice/crio-f6d1fbee5256aa68319ad17ca8b3fbb005b4c5a9959c0361b59b3fe13a0f7132 WatchSource:0}: Error finding container f6d1fbee5256aa68319ad17ca8b3fbb005b4c5a9959c0361b59b3fe13a0f7132: Status 404 returned error can't find the container with id f6d1fbee5256aa68319ad17ca8b3fbb005b4c5a9959c0361b59b3fe13a0f7132
Jul 24 15:17:23.012867 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:23.012781    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624088332016" event={"ID":"904e997b-28fd-43bb-a9bc-b4d3f6940040","Type":"ContainerStarted","Data":"f6d1fbee5256aa68319ad17ca8b3fbb005b4c5a9959c0361b59b3fe13a0f7132"}
Jul 24 15:17:23.014006 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:23.013986    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a3723f61edf50006569f3f0fd06906e7c38e0c04e0d883e5495dc8baa826f81"
Jul 24 15:17:23.014119 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:23.014016    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082966832"
Jul 24 15:17:23.024298 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:17:23.024273    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7459-140624082966832\" is forbidden: User \"system:node:ip-10-0-143-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-246.ec2.internal' and this object" podUID="b273463b-551b-486d-8882-715a2b83442a" pod="opendatahub/test-curl-7459-140624082966832"
Jul 24 15:17:23.333847 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:17:23.333769    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7459-140624082966832\" is forbidden: User \"system:node:ip-10-0-143-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-246.ec2.internal' and this object" podUID="b273463b-551b-486d-8882-715a2b83442a" pod="opendatahub/test-curl-7459-140624082966832"
Jul 24 15:17:23.335347 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:23.335325    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b273463b-551b-486d-8882-715a2b83442a" path="/var/lib/kubelet/pods/b273463b-551b-486d-8882-715a2b83442a/volumes"
Jul 24 15:17:24.019663 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:24.019629    2581 generic.go:356] "Generic (PLEG): container finished" podID="904e997b-28fd-43bb-a9bc-b4d3f6940040" containerID="90faa87a9439f843c96a6edadbfdf09fe713b0a2a9935a3263425f8fbdf43980" exitCode=0
Jul 24 15:17:24.019845 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:24.019671    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624088332016" event={"ID":"904e997b-28fd-43bb-a9bc-b4d3f6940040","Type":"ContainerDied","Data":"90faa87a9439f843c96a6edadbfdf09fe713b0a2a9935a3263425f8fbdf43980"}
Jul 24 15:17:25.151980 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.151953    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:25.205529 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.205504    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7459-140624088332016_904e997b-28fd-43bb-a9bc-b4d3f6940040/test-curl-7459-140624088332016/0.log"
Jul 24 15:17:25.260933 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.260904    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqqt2\" (UniqueName: \"kubernetes.io/projected/904e997b-28fd-43bb-a9bc-b4d3f6940040-kube-api-access-pqqt2\") pod \"904e997b-28fd-43bb-a9bc-b4d3f6940040\" (UID: \"904e997b-28fd-43bb-a9bc-b4d3f6940040\") "
Jul 24 15:17:25.261599 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.261580    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7459-140624088332016"]
Jul 24 15:17:25.262987 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.262959    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/904e997b-28fd-43bb-a9bc-b4d3f6940040-kube-api-access-pqqt2" (OuterVolumeSpecName: "kube-api-access-pqqt2") pod "904e997b-28fd-43bb-a9bc-b4d3f6940040" (UID: "904e997b-28fd-43bb-a9bc-b4d3f6940040"). InnerVolumeSpecName "kube-api-access-pqqt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:17:25.265642 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.265611    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7459-140624088332016"]
Jul 24 15:17:25.334597 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.334517    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="904e997b-28fd-43bb-a9bc-b4d3f6940040" path="/var/lib/kubelet/pods/904e997b-28fd-43bb-a9bc-b4d3f6940040/volumes"
Jul 24 15:17:25.361809 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.361777    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqqt2\" (UniqueName: \"kubernetes.io/projected/904e997b-28fd-43bb-a9bc-b4d3f6940040-kube-api-access-pqqt2\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:25.387888 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.387860    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7459-140624088332016"]
Jul 24 15:17:25.388231 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.388217    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="904e997b-28fd-43bb-a9bc-b4d3f6940040" containerName="test-curl-7459-140624088332016"
Jul 24 15:17:25.388280 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.388233    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="904e997b-28fd-43bb-a9bc-b4d3f6940040" containerName="test-curl-7459-140624088332016"
Jul 24 15:17:25.388315 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.388303    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="904e997b-28fd-43bb-a9bc-b4d3f6940040" containerName="test-curl-7459-140624088332016"
Jul 24 15:17:25.391511 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.391496    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:25.400308 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.400287    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624088332016"]
Jul 24 15:17:25.462368 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.462326    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrgdw\" (UniqueName: \"kubernetes.io/projected/0936e091-4012-4202-b67c-3f858e3f2e4e-kube-api-access-jrgdw\") pod \"test-curl-7459-140624088332016\" (UID: \"0936e091-4012-4202-b67c-3f858e3f2e4e\") " pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:25.563407 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.563369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrgdw\" (UniqueName: \"kubernetes.io/projected/0936e091-4012-4202-b67c-3f858e3f2e4e-kube-api-access-jrgdw\") pod \"test-curl-7459-140624088332016\" (UID: \"0936e091-4012-4202-b67c-3f858e3f2e4e\") " pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:25.572944 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.572914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrgdw\" (UniqueName: \"kubernetes.io/projected/0936e091-4012-4202-b67c-3f858e3f2e4e-kube-api-access-jrgdw\") pod \"test-curl-7459-140624088332016\" (UID: \"0936e091-4012-4202-b67c-3f858e3f2e4e\") " pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:25.701110 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.701055    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:25.829640 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:25.829615    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624088332016"]
Jul 24 15:17:25.831477 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:17:25.831450    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0936e091_4012_4202_b67c_3f858e3f2e4e.slice/crio-b86c7ef06885d94a1da8c813dc24ebf808462207c17c290f8596528c34e024d3 WatchSource:0}: Error finding container b86c7ef06885d94a1da8c813dc24ebf808462207c17c290f8596528c34e024d3: Status 404 returned error can't find the container with id b86c7ef06885d94a1da8c813dc24ebf808462207c17c290f8596528c34e024d3
Jul 24 15:17:26.031005 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:26.030920    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:26.031005 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:26.030940    2581 scope.go:117] "RemoveContainer" containerID="90faa87a9439f843c96a6edadbfdf09fe713b0a2a9935a3263425f8fbdf43980"
Jul 24 15:17:26.032295 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:26.032270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624088332016" event={"ID":"0936e091-4012-4202-b67c-3f858e3f2e4e","Type":"ContainerStarted","Data":"b86c7ef06885d94a1da8c813dc24ebf808462207c17c290f8596528c34e024d3"}
Jul 24 15:17:27.039165 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:27.039131    2581 generic.go:356] "Generic (PLEG): container finished" podID="0936e091-4012-4202-b67c-3f858e3f2e4e" containerID="07ad3cd41d2c773959aa035ffe634100bc1601307cf5c8797ec4e5b7db0ed364" exitCode=0
Jul 24 15:17:27.039583 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:27.039199    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624088332016" event={"ID":"0936e091-4012-4202-b67c-3f858e3f2e4e","Type":"ContainerDied","Data":"07ad3cd41d2c773959aa035ffe634100bc1601307cf5c8797ec4e5b7db0ed364"}
Jul 24 15:17:28.181277 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.181253    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:28.219074 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.219029    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7459-140624088332016_0936e091-4012-4202-b67c-3f858e3f2e4e/test-curl-7459-140624088332016/0.log"
Jul 24 15:17:28.246360 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.246327    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7459-140624088332016"]
Jul 24 15:17:28.260503 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.260475    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7459-140624088332016"]
Jul 24 15:17:28.288959 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.288934    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrgdw\" (UniqueName: \"kubernetes.io/projected/0936e091-4012-4202-b67c-3f858e3f2e4e-kube-api-access-jrgdw\") pod \"0936e091-4012-4202-b67c-3f858e3f2e4e\" (UID: \"0936e091-4012-4202-b67c-3f858e3f2e4e\") "
Jul 24 15:17:28.290934 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.290913    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0936e091-4012-4202-b67c-3f858e3f2e4e-kube-api-access-jrgdw" (OuterVolumeSpecName: "kube-api-access-jrgdw") pod "0936e091-4012-4202-b67c-3f858e3f2e4e" (UID: "0936e091-4012-4202-b67c-3f858e3f2e4e"). InnerVolumeSpecName "kube-api-access-jrgdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:17:28.390316 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.390269    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrgdw\" (UniqueName: \"kubernetes.io/projected/0936e091-4012-4202-b67c-3f858e3f2e4e-kube-api-access-jrgdw\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:28.433207 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.433178    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7459-140624082904528"]
Jul 24 15:17:28.433549 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.433537    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0936e091-4012-4202-b67c-3f858e3f2e4e" containerName="test-curl-7459-140624088332016"
Jul 24 15:17:28.433597 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.433551    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0936e091-4012-4202-b67c-3f858e3f2e4e" containerName="test-curl-7459-140624088332016"
Jul 24 15:17:28.433635 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.433602    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0936e091-4012-4202-b67c-3f858e3f2e4e" containerName="test-curl-7459-140624088332016"
Jul 24 15:17:28.436585 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.436569    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:28.491346 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.491320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnnvt\" (UniqueName: \"kubernetes.io/projected/7bada7a6-d905-4e25-be0e-9c62cc81488f-kube-api-access-tnnvt\") pod \"test-curl-7459-140624082904528\" (UID: \"7bada7a6-d905-4e25-be0e-9c62cc81488f\") " pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:28.500036 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.500007    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624082904528"]
Jul 24 15:17:28.592426 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.592342    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnnvt\" (UniqueName: \"kubernetes.io/projected/7bada7a6-d905-4e25-be0e-9c62cc81488f-kube-api-access-tnnvt\") pod \"test-curl-7459-140624082904528\" (UID: \"7bada7a6-d905-4e25-be0e-9c62cc81488f\") " pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:28.606637 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.606609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnnvt\" (UniqueName: \"kubernetes.io/projected/7bada7a6-d905-4e25-be0e-9c62cc81488f-kube-api-access-tnnvt\") pod \"test-curl-7459-140624082904528\" (UID: \"7bada7a6-d905-4e25-be0e-9c62cc81488f\") " pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:28.745329 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.745294    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:28.883957 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.883929    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624082904528"]
Jul 24 15:17:28.885033 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:17:28.885008    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bada7a6_d905_4e25_be0e_9c62cc81488f.slice/crio-6914472ad37f7f260da0bbef22cf6bdbe1b550d37f7c44b14fd5ff338fdbabd3 WatchSource:0}: Error finding container 6914472ad37f7f260da0bbef22cf6bdbe1b550d37f7c44b14fd5ff338fdbabd3: Status 404 returned error can't find the container with id 6914472ad37f7f260da0bbef22cf6bdbe1b550d37f7c44b14fd5ff338fdbabd3
Jul 24 15:17:28.996587 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:28.996544    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n" podUID="5b30b3c6-1d55-4883-a085-30ece79b4384" containerName="main" containerID="cri-o://1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e" gracePeriod=45
Jul 24 15:17:29.049969 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.049939    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624082904528" event={"ID":"7bada7a6-d905-4e25-be0e-9c62cc81488f","Type":"ContainerStarted","Data":"6914472ad37f7f260da0bbef22cf6bdbe1b550d37f7c44b14fd5ff338fdbabd3"}
Jul 24 15:17:29.051231 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.051210    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b86c7ef06885d94a1da8c813dc24ebf808462207c17c290f8596528c34e024d3"
Jul 24 15:17:29.051327 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.051252    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:29.061753 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:17:29.061727    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7459-140624088332016\" is forbidden: User \"system:node:ip-10-0-143-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-246.ec2.internal' and this object" podUID="0936e091-4012-4202-b67c-3f858e3f2e4e" pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:29.262639 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.262598    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:17:29.264853 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:17:29.264825    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7459-140624088332016\" is forbidden: User \"system:node:ip-10-0-143-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-246.ec2.internal' and this object" podUID="0936e091-4012-4202-b67c-3f858e3f2e4e" pod="opendatahub/test-curl-7459-140624088332016"
Jul 24 15:17:29.299951 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.299920    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b30b3c6-1d55-4883-a085-30ece79b4384-tls-certs\") pod \"5b30b3c6-1d55-4883-a085-30ece79b4384\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") "
Jul 24 15:17:29.300150 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.299972    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-tmp-dir\") pod \"5b30b3c6-1d55-4883-a085-30ece79b4384\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") "
Jul 24 15:17:29.300150 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.300000    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-home\") pod \"5b30b3c6-1d55-4883-a085-30ece79b4384\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") "
Jul 24 15:17:29.300289 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.300152    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flzxb\" (UniqueName: \"kubernetes.io/projected/5b30b3c6-1d55-4883-a085-30ece79b4384-kube-api-access-flzxb\") pod \"5b30b3c6-1d55-4883-a085-30ece79b4384\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") "
Jul 24 15:17:29.300289 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.300234    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-dshm\") pod \"5b30b3c6-1d55-4883-a085-30ece79b4384\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") "
Jul 24 15:17:29.300289 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.300245    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-home" (OuterVolumeSpecName: "home") pod "5b30b3c6-1d55-4883-a085-30ece79b4384" (UID: "5b30b3c6-1d55-4883-a085-30ece79b4384"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:17:29.300289 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.300265    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5b30b3c6-1d55-4883-a085-30ece79b4384" (UID: "5b30b3c6-1d55-4883-a085-30ece79b4384"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:17:29.300289 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.300274    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-model-cache\") pod \"5b30b3c6-1d55-4883-a085-30ece79b4384\" (UID: \"5b30b3c6-1d55-4883-a085-30ece79b4384\") "
Jul 24 15:17:29.300555 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.300530    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-model-cache" (OuterVolumeSpecName: "model-cache") pod "5b30b3c6-1d55-4883-a085-30ece79b4384" (UID: "5b30b3c6-1d55-4883-a085-30ece79b4384"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:17:29.300656 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.300640    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:29.300717 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.300660    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:29.300717 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.300673    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:29.302308 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.302281    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b30b3c6-1d55-4883-a085-30ece79b4384-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5b30b3c6-1d55-4883-a085-30ece79b4384" (UID: "5b30b3c6-1d55-4883-a085-30ece79b4384"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:17:29.302442 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.302308    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b30b3c6-1d55-4883-a085-30ece79b4384-kube-api-access-flzxb" (OuterVolumeSpecName: "kube-api-access-flzxb") pod "5b30b3c6-1d55-4883-a085-30ece79b4384" (UID: "5b30b3c6-1d55-4883-a085-30ece79b4384"). InnerVolumeSpecName "kube-api-access-flzxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:17:29.302442 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.302365    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-dshm" (OuterVolumeSpecName: "dshm") pod "5b30b3c6-1d55-4883-a085-30ece79b4384" (UID: "5b30b3c6-1d55-4883-a085-30ece79b4384"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:17:29.335437 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.335406    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0936e091-4012-4202-b67c-3f858e3f2e4e" path="/var/lib/kubelet/pods/0936e091-4012-4202-b67c-3f858e3f2e4e/volumes"
Jul 24 15:17:29.401328 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.401300    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b30b3c6-1d55-4883-a085-30ece79b4384-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:29.401328 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.401327    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flzxb\" (UniqueName: \"kubernetes.io/projected/5b30b3c6-1d55-4883-a085-30ece79b4384-kube-api-access-flzxb\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:29.401546 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.401339    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b30b3c6-1d55-4883-a085-30ece79b4384-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:29.819683 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:29.819643    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq" podUID="bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" containerName="main" containerID="cri-o://292402792ab8f5ec967798283dbffeea82f28298d134de27469db6d4d4ccc45e" gracePeriod=45
Jul 24 15:17:30.058930 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.058896    2581 generic.go:356] "Generic (PLEG): container finished" podID="7bada7a6-d905-4e25-be0e-9c62cc81488f" containerID="326851ada5a3df94d14b5d362bb73852f5229cc0026df36ec0a52cb9a3f9ca18" exitCode=0
Jul 24 15:17:30.059124 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.058988    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624082904528" event={"ID":"7bada7a6-d905-4e25-be0e-9c62cc81488f","Type":"ContainerDied","Data":"326851ada5a3df94d14b5d362bb73852f5229cc0026df36ec0a52cb9a3f9ca18"}
Jul 24 15:17:30.060341 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.060316    2581 generic.go:356] "Generic (PLEG): container finished" podID="5b30b3c6-1d55-4883-a085-30ece79b4384" containerID="1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e" exitCode=0
Jul 24 15:17:30.060454 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.060352    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"
Jul 24 15:17:30.060454 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.060396    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n" event={"ID":"5b30b3c6-1d55-4883-a085-30ece79b4384","Type":"ContainerDied","Data":"1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e"}
Jul 24 15:17:30.060454 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.060426    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n" event={"ID":"5b30b3c6-1d55-4883-a085-30ece79b4384","Type":"ContainerDied","Data":"3340e91da11300401f9a5bf2ac0f3fed13a255bb983536f1118b7cc02d92d6ac"}
Jul 24 15:17:30.060454 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.060449    2581 scope.go:117] "RemoveContainer" containerID="1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e"
Jul 24 15:17:30.062332 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.062311    2581 generic.go:356] "Generic (PLEG): container finished" podID="bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" containerID="292402792ab8f5ec967798283dbffeea82f28298d134de27469db6d4d4ccc45e" exitCode=0
Jul 24 15:17:30.062432 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.062381    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq" event={"ID":"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff","Type":"ContainerDied","Data":"292402792ab8f5ec967798283dbffeea82f28298d134de27469db6d4d4ccc45e"}
Jul 24 15:17:30.070284 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.070268    2581 scope.go:117] "RemoveContainer" containerID="1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e"
Jul 24 15:17:30.070477 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:17:30.070458    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e\": container with ID starting with 1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e not found: ID does not exist" containerID="1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e"
Jul 24 15:17:30.070531 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.070487    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e"} err="failed to get container status \"1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e\": rpc error: code = NotFound desc = could not find container \"1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e\": container with ID starting with 1188c6d4ff553441339a9766cd8dd56da3f230cb547e438729822252ff077a1e not found: ID does not exist"
Jul 24 15:17:30.075222 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.075204    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:17:30.107049 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107021    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tmp-dir\") pod \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") "
Jul 24 15:17:30.107221 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107080    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-model-cache\") pod \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") "
Jul 24 15:17:30.107221 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107127    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hqpn\" (UniqueName: \"kubernetes.io/projected/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-kube-api-access-5hqpn\") pod \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") "
Jul 24 15:17:30.107221 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107152    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-home\") pod \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") "
Jul 24 15:17:30.107221 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107191    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-dshm\") pod \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") "
Jul 24 15:17:30.107417 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107319    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tls-certs\") pod \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\" (UID: \"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff\") "
Jul 24 15:17:30.107417 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107356    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" (UID: "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:17:30.107417 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107390    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-model-cache" (OuterVolumeSpecName: "model-cache") pod "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" (UID: "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:17:30.107571 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107556    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-home" (OuterVolumeSpecName: "home") pod "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" (UID: "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:17:30.107640 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107624    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:30.107703 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107646    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:30.107703 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.107658    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:30.109392 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.109361    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-dshm" (OuterVolumeSpecName: "dshm") pod "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" (UID: "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:17:30.109514 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.109497    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-kube-api-access-5hqpn" (OuterVolumeSpecName: "kube-api-access-5hqpn") pod "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" (UID: "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff"). InnerVolumeSpecName "kube-api-access-5hqpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:17:30.109564 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.109529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" (UID: "bbe4aac8-424a-4242-bf6e-0dccbf2af7ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:17:30.115944 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.115916    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"]
Jul 24 15:17:30.119651 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.119633    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-dc7d81/rate-test-model-dc7d81-kserve-5fffbc587-d9g7n"]
Jul 24 15:17:30.208402 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.208367    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hqpn\" (UniqueName: \"kubernetes.io/projected/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-kube-api-access-5hqpn\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:30.208402 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.208395    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:30.208402 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:30.208408    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:31.068627 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.068600    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"
Jul 24 15:17:31.069064 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.068648    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq" event={"ID":"bbe4aac8-424a-4242-bf6e-0dccbf2af7ff","Type":"ContainerDied","Data":"62fb1acd5fcc4fc7a2e47c2fd6ac678e4bd96169f9bc2000efac9d7962c78eb9"}
Jul 24 15:17:31.069064 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.068706    2581 scope.go:117] "RemoveContainer" containerID="292402792ab8f5ec967798283dbffeea82f28298d134de27469db6d4d4ccc45e"
Jul 24 15:17:31.099655 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.099623    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"]
Jul 24 15:17:31.105273 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.105245    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8ea489/rate-test-model-8ea489-kserve-5c7c654b55-dhqqq"]
Jul 24 15:17:31.216136 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.216112    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:31.240356 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.240324    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7459-140624082904528_7bada7a6-d905-4e25-be0e-9c62cc81488f/test-curl-7459-140624082904528/0.log"
Jul 24 15:17:31.270290 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.270255    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7459-140624082904528"]
Jul 24 15:17:31.275403 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.275378    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7459-140624082904528"]
Jul 24 15:17:31.317572 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.317538    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnnvt\" (UniqueName: \"kubernetes.io/projected/7bada7a6-d905-4e25-be0e-9c62cc81488f-kube-api-access-tnnvt\") pod \"7bada7a6-d905-4e25-be0e-9c62cc81488f\" (UID: \"7bada7a6-d905-4e25-be0e-9c62cc81488f\") "
Jul 24 15:17:31.319580 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.319518    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bada7a6-d905-4e25-be0e-9c62cc81488f-kube-api-access-tnnvt" (OuterVolumeSpecName: "kube-api-access-tnnvt") pod "7bada7a6-d905-4e25-be0e-9c62cc81488f" (UID: "7bada7a6-d905-4e25-be0e-9c62cc81488f"). InnerVolumeSpecName "kube-api-access-tnnvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:17:31.335276 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.335251    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b30b3c6-1d55-4883-a085-30ece79b4384" path="/var/lib/kubelet/pods/5b30b3c6-1d55-4883-a085-30ece79b4384/volumes"
Jul 24 15:17:31.335574 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.335562    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bada7a6-d905-4e25-be0e-9c62cc81488f" path="/var/lib/kubelet/pods/7bada7a6-d905-4e25-be0e-9c62cc81488f/volumes"
Jul 24 15:17:31.335909 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.335896    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" path="/var/lib/kubelet/pods/bbe4aac8-424a-4242-bf6e-0dccbf2af7ff/volumes"
Jul 24 15:17:31.419237 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.419197    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnnvt\" (UniqueName: \"kubernetes.io/projected/7bada7a6-d905-4e25-be0e-9c62cc81488f-kube-api-access-tnnvt\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:31.419663 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.419642    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7459-140624082904528"]
Jul 24 15:17:31.419999 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.419986    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b30b3c6-1d55-4883-a085-30ece79b4384" containerName="main"
Jul 24 15:17:31.420051 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.420001    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b30b3c6-1d55-4883-a085-30ece79b4384" containerName="main"
Jul 24 15:17:31.420051 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.420018    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" containerName="main"
Jul 24 15:17:31.420051 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.420024    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" containerName="main"
Jul 24 15:17:31.420051 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.420029    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bada7a6-d905-4e25-be0e-9c62cc81488f" containerName="test-curl-7459-140624082904528"
Jul 24 15:17:31.420051 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.420035    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bada7a6-d905-4e25-be0e-9c62cc81488f" containerName="test-curl-7459-140624082904528"
Jul 24 15:17:31.420241 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.420114    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bada7a6-d905-4e25-be0e-9c62cc81488f" containerName="test-curl-7459-140624082904528"
Jul 24 15:17:31.420241 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.420122    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b30b3c6-1d55-4883-a085-30ece79b4384" containerName="main"
Jul 24 15:17:31.420241 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.420136    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe4aac8-424a-4242-bf6e-0dccbf2af7ff" containerName="main"
Jul 24 15:17:31.424383 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.424366    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:31.433208 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.433182    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624082904528"]
Jul 24 15:17:31.520056 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.520019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx7jg\" (UniqueName: \"kubernetes.io/projected/e89767ba-7fe3-4f2d-b96c-13bf095e6f6c-kube-api-access-jx7jg\") pod \"test-curl-7459-140624082904528\" (UID: \"e89767ba-7fe3-4f2d-b96c-13bf095e6f6c\") " pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:31.620531 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.620502    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx7jg\" (UniqueName: \"kubernetes.io/projected/e89767ba-7fe3-4f2d-b96c-13bf095e6f6c-kube-api-access-jx7jg\") pod \"test-curl-7459-140624082904528\" (UID: \"e89767ba-7fe3-4f2d-b96c-13bf095e6f6c\") " pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:31.631780 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.631739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx7jg\" (UniqueName: \"kubernetes.io/projected/e89767ba-7fe3-4f2d-b96c-13bf095e6f6c-kube-api-access-jx7jg\") pod \"test-curl-7459-140624082904528\" (UID: \"e89767ba-7fe3-4f2d-b96c-13bf095e6f6c\") " pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:31.733620 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.733583    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:31.866640 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:31.866615    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7459-140624082904528"]
Jul 24 15:17:31.868984 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:17:31.868956    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode89767ba_7fe3_4f2d_b96c_13bf095e6f6c.slice/crio-6c2147a34ce85322ae320f515d66ae9a95f2cd431d7d95face231f2b08ec47b0 WatchSource:0}: Error finding container 6c2147a34ce85322ae320f515d66ae9a95f2cd431d7d95face231f2b08ec47b0: Status 404 returned error can't find the container with id 6c2147a34ce85322ae320f515d66ae9a95f2cd431d7d95face231f2b08ec47b0
Jul 24 15:17:32.075038 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:32.075011    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:32.075038 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:32.075036    2581 scope.go:117] "RemoveContainer" containerID="326851ada5a3df94d14b5d362bb73852f5229cc0026df36ec0a52cb9a3f9ca18"
Jul 24 15:17:32.076395 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:32.076370    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624082904528" event={"ID":"e89767ba-7fe3-4f2d-b96c-13bf095e6f6c","Type":"ContainerStarted","Data":"6c2147a34ce85322ae320f515d66ae9a95f2cd431d7d95face231f2b08ec47b0"}
Jul 24 15:17:33.083947 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:33.083914    2581 generic.go:356] "Generic (PLEG): container finished" podID="e89767ba-7fe3-4f2d-b96c-13bf095e6f6c" containerID="8ea18ff0144bb6af45a4e1c91b1ed0cc03317bd4f96a0406141779caa8a3a16e" exitCode=0
Jul 24 15:17:33.084344 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:33.083978    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7459-140624082904528" event={"ID":"e89767ba-7fe3-4f2d-b96c-13bf095e6f6c","Type":"ContainerDied","Data":"8ea18ff0144bb6af45a4e1c91b1ed0cc03317bd4f96a0406141779caa8a3a16e"}
Jul 24 15:17:34.223715 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:34.223689    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:34.244173 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:34.244148    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7459-140624082904528_e89767ba-7fe3-4f2d-b96c-13bf095e6f6c/test-curl-7459-140624082904528/0.log"
Jul 24 15:17:34.265887 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:34.265859    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7459-140624082904528"]
Jul 24 15:17:34.270246 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:34.270224    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7459-140624082904528"]
Jul 24 15:17:34.341767 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:34.341685    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jx7jg\" (UniqueName: \"kubernetes.io/projected/e89767ba-7fe3-4f2d-b96c-13bf095e6f6c-kube-api-access-jx7jg\") pod \"e89767ba-7fe3-4f2d-b96c-13bf095e6f6c\" (UID: \"e89767ba-7fe3-4f2d-b96c-13bf095e6f6c\") "
Jul 24 15:17:34.343683 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:34.343656    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e89767ba-7fe3-4f2d-b96c-13bf095e6f6c-kube-api-access-jx7jg" (OuterVolumeSpecName: "kube-api-access-jx7jg") pod "e89767ba-7fe3-4f2d-b96c-13bf095e6f6c" (UID: "e89767ba-7fe3-4f2d-b96c-13bf095e6f6c"). InnerVolumeSpecName "kube-api-access-jx7jg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:17:34.442523 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:34.442492    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx7jg\" (UniqueName: \"kubernetes.io/projected/e89767ba-7fe3-4f2d-b96c-13bf095e6f6c-kube-api-access-jx7jg\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:17:34.802647 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:34.802614    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:35.097277 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.097200    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:35.097277 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.097259    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c2147a34ce85322ae320f515d66ae9a95f2cd431d7d95face231f2b08ec47b0"
Jul 24 15:17:35.107206 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:17:35.107180    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7459-140624082904528\" is forbidden: User \"system:node:ip-10-0-143-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-246.ec2.internal' and this object" podUID="e89767ba-7fe3-4f2d-b96c-13bf095e6f6c" pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:35.132548 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.132525    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"]
Jul 24 15:17:35.132875 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.132863    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e89767ba-7fe3-4f2d-b96c-13bf095e6f6c" containerName="test-curl-7459-140624082904528"
Jul 24 15:17:35.132914 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.132876    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e89767ba-7fe3-4f2d-b96c-13bf095e6f6c" containerName="test-curl-7459-140624082904528"
Jul 24 15:17:35.132958 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.132939    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e89767ba-7fe3-4f2d-b96c-13bf095e6f6c" containerName="test-curl-7459-140624082904528"
Jul 24 15:17:35.135800 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.135786    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.138465 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:17:35.138437    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7459-140624082904528\" is forbidden: User \"system:node:ip-10-0-143-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-246.ec2.internal' and this object" podUID="e89767ba-7fe3-4f2d-b96c-13bf095e6f6c" pod="opendatahub/test-curl-7459-140624082904528"
Jul 24 15:17:35.138647 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.138632    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 15:17:35.138698 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.138657    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-bb95eb-openshift-default-dockercfg-t8vv9\""
Jul 24 15:17:35.152238 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.152217    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"]
Jul 24 15:17:35.170670 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.170646    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:35.248862 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.248827    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8g9n\" (UniqueName: \"kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-kube-api-access-n8g9n\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.249295 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.248884    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-certs\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.249295 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.248910    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-data\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.249295 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.248940    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istiod-ca-cert\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.249295 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.248963    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-socket\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.249295 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.249018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-podinfo\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.249295 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.249040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-credential-socket\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.249295 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.249056    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-token\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.249295 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.249073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-envoy\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.267572 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.267540    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:35.333869 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.333838    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e89767ba-7fe3-4f2d-b96c-13bf095e6f6c" path="/var/lib/kubelet/pods/e89767ba-7fe3-4f2d-b96c-13bf095e6f6c/volumes"
Jul 24 15:17:35.350340 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350263    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-certs\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.350340 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-data\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.350493 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istiod-ca-cert\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.350493 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350464    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-socket\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.350624 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350600    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-podinfo\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.350690 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350653    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-credential-socket\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.350690 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-token\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.350784 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350700    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-data\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.350784 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-envoy\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.350784 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350719    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-certs\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.350784 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350757    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-socket\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.350983 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8g9n\" (UniqueName: \"kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-kube-api-access-n8g9n\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.351037 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.350991    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-credential-socket\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.351198 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.351181    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istiod-ca-cert\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.352945 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.352927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-envoy\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.353219 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.353193    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-podinfo\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.362465 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.362445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-token\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.362705 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.362687    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8g9n\" (UniqueName: \"kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-kube-api-access-n8g9n\") pod \"e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") " pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.445459 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.445432    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:35.583886 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:35.583860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"]
Jul 24 15:17:35.585907 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:17:35.585885    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c7079eb_c2b0_4ad6_9296_66a4653c50d8.slice/crio-191f1715a97e90021bd6fdfc2b016593ac712c63778e3fce56f6f1c001c2ead0 WatchSource:0}: Error finding container 191f1715a97e90021bd6fdfc2b016593ac712c63778e3fce56f6f1c001c2ead0: Status 404 returned error can't find the container with id 191f1715a97e90021bd6fdfc2b016593ac712c63778e3fce56f6f1c001c2ead0
Jul 24 15:17:36.106261 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:36.106225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" event={"ID":"6c7079eb-c2b0-4ad6-9296-66a4653c50d8","Type":"ContainerStarted","Data":"191f1715a97e90021bd6fdfc2b016593ac712c63778e3fce56f6f1c001c2ead0"}
Jul 24 15:17:38.191828 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:38.191786    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:17:38.192121 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:38.191863    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:17:38.192121 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:38.191892    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:17:39.126350 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:39.126316    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" event={"ID":"6c7079eb-c2b0-4ad6-9296-66a4653c50d8","Type":"ContainerStarted","Data":"ff646259ceeda69328fd02a247b22b7e82cb5c6d83d22414feec641c935fe767"}
Jul 24 15:17:39.152401 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:39.152355    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" podStartSLOduration=1.548890637 podStartE2EDuration="4.152340436s" totalImagesPullingTime="2.603449799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:17:39.152076094 +0000 UTC m=+2737.655794063" watchObservedRunningTime="2026-07-24 15:17:39.152340436 +0000 UTC m=+2737.656058394"
Jul 24 15:17:39.446067 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:39.445992    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:39.451142 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:39.451118    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:40.131598 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:40.131565    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:40.132558 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:40.132540    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:41.296629 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:41.296596    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:41.464380 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:41.464346    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:42.221931 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:42.221902    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:42.490761 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:42.490674    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:42.749117 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:42.749006    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:42.842328 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:42.842278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:45.651107 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:45.651064    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:46.701551 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:46.701520    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:17:55.132886 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:55.132850    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 24 15:17:55.133338 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:55.132905    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 24 15:17:55.210745 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:55.210714    2581 generic.go:356] "Generic (PLEG): container finished" podID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerID="ff646259ceeda69328fd02a247b22b7e82cb5c6d83d22414feec641c935fe767" exitCode=0
Jul 24 15:17:55.210921 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:55.210785    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" event={"ID":"6c7079eb-c2b0-4ad6-9296-66a4653c50d8","Type":"ContainerDied","Data":"ff646259ceeda69328fd02a247b22b7e82cb5c6d83d22414feec641c935fe767"}
Jul 24 15:17:55.211211 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:55.211194    2581 scope.go:117] "RemoveContainer" containerID="ff646259ceeda69328fd02a247b22b7e82cb5c6d83d22414feec641c935fe767"
Jul 24 15:17:55.213344 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:55.213316    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:17:55.213410 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:55.213378    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:17:55.213447 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:55.213409    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:17:56.218334 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:56.218298    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" event={"ID":"6c7079eb-c2b0-4ad6-9296-66a4653c50d8","Type":"ContainerStarted","Data":"3c08bc3a8e024b043f580a0f1af7c0d3c7ad6e3c30827569a8b6cdd5f6182972"}
Jul 24 15:17:56.446873 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:56.446164    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:17:56.446873 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:56.446574    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 24 15:17:56.446873 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:56.446621    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 24 15:17:56.605201 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:56.605118    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:57.051855 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:57.051818    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:57.220616 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:57.220576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:57.446891 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:57.446854    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 24 15:17:57.447197 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:57.447165    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 24 15:17:58.446630 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:58.446587    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 24 15:17:58.447139 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:58.446651    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 24 15:17:59.446713 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:59.446678    2581 patch_prober.go:28] interesting pod/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 24 15:17:59.447356 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:59.447324    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 24 15:17:59.711451 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:59.711359    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:17:59.807556 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:17:59.807522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:18:00.450354 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:00.450324    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:18:00.450840 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:00.450687    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:18:00.451295 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:00.451277    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:18:01.561897 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:01.561860    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:18:01.892836 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:01.892797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:18:01.996268 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:01.996226    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:18:02.128807 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:02.128750    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:18:04.240689 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:04.240635    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:18:08.709392 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:08.709355    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:18:33.211169 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.211136    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"]
Jul 24 15:18:33.214618 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.214599    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.218330 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.218306    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-bb95eb\"/\"ipp-route-bb95eb-kserve-self-signed-certs\""
Jul 24 15:18:33.218459 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.218306    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-bb95eb\"/\"default-dockercfg-xwlpc\""
Jul 24 15:18:33.218459 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.218343    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-bb95eb\"/\"kube-root-ca.crt\""
Jul 24 15:18:33.218459 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.218310    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-bb95eb\"/\"openshift-service-ca.crt\""
Jul 24 15:18:33.225428 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.225406    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"]
Jul 24 15:18:33.297515 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.297481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tmp-dir\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.297683 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.297526    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-model-cache\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.297683 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.297550    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6rn5\" (UniqueName: \"kubernetes.io/projected/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-kube-api-access-q6rn5\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.297683 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.297642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tls-certs\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.297683 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.297669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-home\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.297825 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.297685    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-dshm\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.305169 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.305128    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:18:33.398414 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.398385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tmp-dir\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.398608 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.398423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-model-cache\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.398608 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.398444    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6rn5\" (UniqueName: \"kubernetes.io/projected/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-kube-api-access-q6rn5\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.398608 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.398487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tls-certs\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.398608 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.398503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-home\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.398608 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.398516    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-dshm\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.399222 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.399195    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tmp-dir\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.399332 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.399272    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-model-cache\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.399449 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.399424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-home\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.401962 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.401938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tls-certs\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.406984 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.406961    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-dshm\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.407968 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.407948    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6rn5\" (UniqueName: \"kubernetes.io/projected/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-kube-api-access-q6rn5\") pod \"ipp-route-bb95eb-kserve-557ff8d987-trdvg\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") " pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.523653 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.523562    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:33.663223 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:33.663136    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"]
Jul 24 15:18:33.665443 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:18:33.665419    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdde6aef_9ab0_451b_a1a6_27b9e0b208a6.slice/crio-2c646b2e980b8ea5790976d4296e8b955bccfd0a03ebea7cb57599f3237972e2 WatchSource:0}: Error finding container 2c646b2e980b8ea5790976d4296e8b955bccfd0a03ebea7cb57599f3237972e2: Status 404 returned error can't find the container with id 2c646b2e980b8ea5790976d4296e8b955bccfd0a03ebea7cb57599f3237972e2
Jul 24 15:18:34.407190 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:34.407148    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg" event={"ID":"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6","Type":"ContainerStarted","Data":"c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed"}
Jul 24 15:18:34.407190 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:34.407194    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg" event={"ID":"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6","Type":"ContainerStarted","Data":"2c646b2e980b8ea5790976d4296e8b955bccfd0a03ebea7cb57599f3237972e2"}
Jul 24 15:18:34.425602 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:34.425550    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg" podStartSLOduration=1.425535475 podStartE2EDuration="1.425535475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:18:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:18:34.424686492 +0000 UTC m=+2792.928404451" watchObservedRunningTime="2026-07-24 15:18:34.425535475 +0000 UTC m=+2792.929253433"
Jul 24 15:18:36.716105 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:36.716070    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:18:43.524050 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:43.524010    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:43.524050 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:43.524058    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:43.536741 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:43.536710    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:44.470310 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:44.470277    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:18:59.587103 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:18:59.587044    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:04.403577 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:04.403543    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:19:14.856553 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:14.856512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:15.147834 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.147798    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"]
Jul 24 15:19:15.151563 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.151547    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.154694 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.154674    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-2edd1c-openshift-default-dockercfg-t8tpf\""
Jul 24 15:19:15.163908 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.163883    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"]
Jul 24 15:19:15.174154 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.174130    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:15.261479 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.261446    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-envoy\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.261669 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.261489    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-credential-socket\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.261669 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.261580    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-token\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.261669 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.261656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-podinfo\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.261815 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.261704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-certs\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.261815 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.261726    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-data\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.261815 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.261751    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.261815 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.261771    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-socket\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.261983 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.261820    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h752t\" (UniqueName: \"kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-kube-api-access-h752t\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.273040 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.273011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:15.362728 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.362691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h752t\" (UniqueName: \"kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-kube-api-access-h752t\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.362934 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.362748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-envoy\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.362934 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.362917    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-credential-socket\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.363066 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.362960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-token\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.363066 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.363003    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-podinfo\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.363066 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.363024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-certs\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.363066 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.363046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-data\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.363350 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.363074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.363350 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.363126    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-socket\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.363350 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.363329    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-credential-socket\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.363525 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.363381    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-certs\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.363525 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.363438    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-data\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.363681 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.363658    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-socket\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.363806 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.363789    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.365169 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.365147    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-envoy\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.365438 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.365418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-podinfo\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.371657 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.371623    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-token\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.372197 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.372180    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h752t\" (UniqueName: \"kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-kube-api-access-h752t\") pod \"e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") " pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.462076 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.461988    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:15.603236 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.603210    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"]
Jul 24 15:19:15.605106 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:19:15.605065    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f2bc3de_95f5_4a48_b9bd_1647907db60f.slice/crio-5970cdca04c1066d0f351947dd37d80915128b7eb4dd3e43eece3895a3ef335b WatchSource:0}: Error finding container 5970cdca04c1066d0f351947dd37d80915128b7eb4dd3e43eece3895a3ef335b: Status 404 returned error can't find the container with id 5970cdca04c1066d0f351947dd37d80915128b7eb4dd3e43eece3895a3ef335b
Jul 24 15:19:15.607226 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.607198    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:19:15.607291 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.607260    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:19:15.607334 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.607288    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:19:15.616530 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:15.616503    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" event={"ID":"3f2bc3de-95f5-4a48-b9bd-1647907db60f","Type":"ContainerStarted","Data":"5970cdca04c1066d0f351947dd37d80915128b7eb4dd3e43eece3895a3ef335b"}
Jul 24 15:19:16.622781 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:16.622750    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" event={"ID":"3f2bc3de-95f5-4a48-b9bd-1647907db60f","Type":"ContainerStarted","Data":"7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14"}
Jul 24 15:19:16.647086 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:16.647036    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" podStartSLOduration=1.647022378 podStartE2EDuration="1.647022378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:19:16.644335078 +0000 UTC m=+2835.148053048" watchObservedRunningTime="2026-07-24 15:19:16.647022378 +0000 UTC m=+2835.150740336"
Jul 24 15:19:17.462198 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:17.462160    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:17.467963 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:17.467934    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:17.627487 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:17.627453    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:17.628373 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:17.628354    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:18.701419 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:18.701379    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:18.792321 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:18.792288    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:19.837219 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:19.837183    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:21.424546 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:21.424514    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:21.919742 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:21.919708    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:22.016034 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:22.015995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:22.103885 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:22.103855    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:23.109542 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:23.109510    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:30.505745 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:30.505705    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:19:38.734138 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:38.734082    2581 generic.go:356] "Generic (PLEG): container finished" podID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerID="7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14" exitCode=0
Jul 24 15:19:38.734567 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:38.734157    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" event={"ID":"3f2bc3de-95f5-4a48-b9bd-1647907db60f","Type":"ContainerDied","Data":"7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14"}
Jul 24 15:19:38.734567 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:38.734556    2581 scope.go:117] "RemoveContainer" containerID="7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14"
Jul 24 15:19:38.735617 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:38.735591    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:19:38.735666 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:38.735656    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:19:38.735701 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:38.735686    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 24 15:19:39.117083 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.117052    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"]
Jul 24 15:19:39.119653 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.119637    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"
Jul 24 15:19:39.122390 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.122366    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:19:39.122578 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.122561    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:19:39.129309 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.129287    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"]
Jul 24 15:19:39.204777 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.204740    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f3350b4-5918-4cc9-b7e5-50d0887089bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568\" (UID: \"7f3350b4-5918-4cc9-b7e5-50d0887089bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"
Jul 24 15:19:39.305471 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.305435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f3350b4-5918-4cc9-b7e5-50d0887089bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568\" (UID: \"7f3350b4-5918-4cc9-b7e5-50d0887089bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"
Jul 24 15:19:39.306009 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.305990    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f3350b4-5918-4cc9-b7e5-50d0887089bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568\" (UID: \"7f3350b4-5918-4cc9-b7e5-50d0887089bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"
Jul 24 15:19:39.430285 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.430198    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"
Jul 24 15:19:39.610873 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.610847    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"]
Jul 24 15:19:39.612788 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:19:39.612759    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f3350b4_5918_4cc9_b7e5_50d0887089bb.slice/crio-c59c82070b78e71bae4219ee950ed12f174323c14938468db96968d61b00bcfc WatchSource:0}: Error finding container c59c82070b78e71bae4219ee950ed12f174323c14938468db96968d61b00bcfc: Status 404 returned error can't find the container with id c59c82070b78e71bae4219ee950ed12f174323c14938468db96968d61b00bcfc
Jul 24 15:19:39.743185 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.741306    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" event={"ID":"3f2bc3de-95f5-4a48-b9bd-1647907db60f","Type":"ContainerStarted","Data":"9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7"}
Jul 24 15:19:39.751570 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:39.751521    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568" event={"ID":"7f3350b4-5918-4cc9-b7e5-50d0887089bb","Type":"ContainerStarted","Data":"c59c82070b78e71bae4219ee950ed12f174323c14938468db96968d61b00bcfc"}
Jul 24 15:19:40.462682 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:40.462644    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:40.462865 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:40.462769    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 24 15:19:40.462865 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:40.462814    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 24 15:19:40.761924 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:40.761832    2581 generic.go:356] "Generic (PLEG): container finished" podID="7f3350b4-5918-4cc9-b7e5-50d0887089bb" containerID="ccad3947b452c9287b44fa3907eddad119c7d919213372a8ad3347724e46e368" exitCode=0
Jul 24 15:19:40.762765 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:40.762370    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568" event={"ID":"7f3350b4-5918-4cc9-b7e5-50d0887089bb","Type":"ContainerDied","Data":"ccad3947b452c9287b44fa3907eddad119c7d919213372a8ad3347724e46e368"}
Jul 24 15:19:41.463305 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:41.463258    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 24 15:19:41.463533 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:41.463329    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 24 15:19:41.919752 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:41.919729    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"
Jul 24 15:19:42.030046 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:42.029958    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f3350b4-5918-4cc9-b7e5-50d0887089bb-maas-api-service-ca\") pod \"7f3350b4-5918-4cc9-b7e5-50d0887089bb\" (UID: \"7f3350b4-5918-4cc9-b7e5-50d0887089bb\") "
Jul 24 15:19:42.030333 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:42.030311    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f3350b4-5918-4cc9-b7e5-50d0887089bb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f3350b4-5918-4cc9-b7e5-50d0887089bb" (UID: "7f3350b4-5918-4cc9-b7e5-50d0887089bb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:19:42.131717 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:42.131683    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f3350b4-5918-4cc9-b7e5-50d0887089bb-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:19:42.463755 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:42.463327    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 24 15:19:42.463755 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:42.463388    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 24 15:19:42.778957 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:42.778859    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568" event={"ID":"7f3350b4-5918-4cc9-b7e5-50d0887089bb","Type":"ContainerDied","Data":"c59c82070b78e71bae4219ee950ed12f174323c14938468db96968d61b00bcfc"}
Jul 24 15:19:42.779299 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:42.779144    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"
Jul 24 15:19:42.779540 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:42.779284    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c59c82070b78e71bae4219ee950ed12f174323c14938468db96968d61b00bcfc"
Jul 24 15:19:43.463423 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:43.463376    2581 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 24 15:19:43.463971 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:43.463445    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 24 15:19:44.466448 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:44.466414    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:44.466935 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:44.466795    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:44.467426 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:44.467408    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:19:51.984547 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:51.984511    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:52.072910 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:52.072870    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:52.173603 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:52.173566    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:54.195265 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:54.195189    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"]
Jul 24 15:19:54.198039 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:54.198013    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2edd1c-d2568"]
Jul 24 15:19:54.201401 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:54.201372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:54.657512 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:54.657474    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:19:55.340355 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:55.340321    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f3350b4-5918-4cc9-b7e5-50d0887089bb" path="/var/lib/kubelet/pods/7f3350b4-5918-4cc9-b7e5-50d0887089bb/volumes"
Jul 24 15:19:57.006795 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:19:57.006759    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:20:00.649984 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:00.649944    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:20:00.749765 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:00.749731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:20:00.920206 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:00.920128    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"]
Jul 24 15:20:00.920363 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:00.920335    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerName="istio-proxy" containerID="cri-o://9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7" gracePeriod=30
Jul 24 15:20:00.947321 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:00.947278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:20:01.158338 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:01.158304    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:20:04.571500 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.571465    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"]
Jul 24 15:20:04.571899 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.571834    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f3350b4-5918-4cc9-b7e5-50d0887089bb" containerName="revoke-keys"
Jul 24 15:20:04.571899 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.571845    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f3350b4-5918-4cc9-b7e5-50d0887089bb" containerName="revoke-keys"
Jul 24 15:20:04.571976 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.571924    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f3350b4-5918-4cc9-b7e5-50d0887089bb" containerName="revoke-keys"
Jul 24 15:20:04.576064 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.576046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"
Jul 24 15:20:04.578558 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.578532    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:20:04.578805 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.578790    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:20:04.583576 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.583551    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"]
Jul 24 15:20:04.741816 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.741779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17704ad-e506-490f-9cb7-17e515c36e03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t\" (UID: \"d17704ad-e506-490f-9cb7-17e515c36e03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"
Jul 24 15:20:04.842858 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.842768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17704ad-e506-490f-9cb7-17e515c36e03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t\" (UID: \"d17704ad-e506-490f-9cb7-17e515c36e03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"
Jul 24 15:20:04.843428 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.843408    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17704ad-e506-490f-9cb7-17e515c36e03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t\" (UID: \"d17704ad-e506-490f-9cb7-17e515c36e03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"
Jul 24 15:20:04.887566 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:04.887526    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"
Jul 24 15:20:05.021736 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:05.021393    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"]
Jul 24 15:20:05.024761 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:20:05.024733    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17704ad_e506_490f_9cb7_17e515c36e03.slice/crio-24c6a0cf07bd3febbb1a1f60893767dc54838e4339a3bfd8d60dd6970540e50b WatchSource:0}: Error finding container 24c6a0cf07bd3febbb1a1f60893767dc54838e4339a3bfd8d60dd6970540e50b: Status 404 returned error can't find the container with id 24c6a0cf07bd3febbb1a1f60893767dc54838e4339a3bfd8d60dd6970540e50b
Jul 24 15:20:05.897968 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:05.897935    2581 generic.go:356] "Generic (PLEG): container finished" podID="d17704ad-e506-490f-9cb7-17e515c36e03" containerID="e33555a4381a256dc77b08002209c86cd02a74ad44978b20d2837c945c897b39" exitCode=0
Jul 24 15:20:05.898358 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:05.898020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t" event={"ID":"d17704ad-e506-490f-9cb7-17e515c36e03","Type":"ContainerDied","Data":"e33555a4381a256dc77b08002209c86cd02a74ad44978b20d2837c945c897b39"}
Jul 24 15:20:05.898358 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:05.898070    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t" event={"ID":"d17704ad-e506-490f-9cb7-17e515c36e03","Type":"ContainerStarted","Data":"24c6a0cf07bd3febbb1a1f60893767dc54838e4339a3bfd8d60dd6970540e50b"}
Jul 24 15:20:06.175274 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.175248    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:20:06.357666 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.357617    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-socket\") pod \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") "
Jul 24 15:20:06.357666 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.357674    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-token\") pod \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") "
Jul 24 15:20:06.357901 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.357701    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-credential-socket\") pod \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") "
Jul 24 15:20:06.357901 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.357725    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istiod-ca-cert\") pod \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") "
Jul 24 15:20:06.357901 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.357774    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-envoy\") pod \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") "
Jul 24 15:20:06.357901 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.357816    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-data\") pod \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") "
Jul 24 15:20:06.357901 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.357881    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-podinfo\") pod \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") "
Jul 24 15:20:06.358209 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.357900    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f2bc3de-95f5-4a48-b9bd-1647907db60f" (UID: "3f2bc3de-95f5-4a48-b9bd-1647907db60f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:20:06.358209 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.357975    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-certs\") pod \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") "
Jul 24 15:20:06.358209 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.358061    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f2bc3de-95f5-4a48-b9bd-1647907db60f" (UID: "3f2bc3de-95f5-4a48-b9bd-1647907db60f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:20:06.358381 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.358238    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f2bc3de-95f5-4a48-b9bd-1647907db60f" (UID: "3f2bc3de-95f5-4a48-b9bd-1647907db60f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:20:06.358381 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.358298    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h752t\" (UniqueName: \"kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-kube-api-access-h752t\") pod \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\" (UID: \"3f2bc3de-95f5-4a48-b9bd-1647907db60f\") "
Jul 24 15:20:06.358489 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.358436    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f2bc3de-95f5-4a48-b9bd-1647907db60f" (UID: "3f2bc3de-95f5-4a48-b9bd-1647907db60f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:20:06.358690 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.358666    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-socket\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:06.358777 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.358693    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-credential-socket\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:06.358777 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.358708    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istiod-ca-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:06.358777 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.358723    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-workload-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:06.358944 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.358922    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f2bc3de-95f5-4a48-b9bd-1647907db60f" (UID: "3f2bc3de-95f5-4a48-b9bd-1647907db60f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:20:06.360178 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.360156    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f2bc3de-95f5-4a48-b9bd-1647907db60f" (UID: "3f2bc3de-95f5-4a48-b9bd-1647907db60f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:20:06.360299 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.360205    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f2bc3de-95f5-4a48-b9bd-1647907db60f" (UID: "3f2bc3de-95f5-4a48-b9bd-1647907db60f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:20:06.360299 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.360254    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-kube-api-access-h752t" (OuterVolumeSpecName: "kube-api-access-h752t") pod "3f2bc3de-95f5-4a48-b9bd-1647907db60f" (UID: "3f2bc3de-95f5-4a48-b9bd-1647907db60f"). InnerVolumeSpecName "kube-api-access-h752t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:20:06.360299 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.360268    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f2bc3de-95f5-4a48-b9bd-1647907db60f" (UID: "3f2bc3de-95f5-4a48-b9bd-1647907db60f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:20:06.459745 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.459645    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-podinfo\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:06.459745 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.459676    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h752t\" (UniqueName: \"kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-kube-api-access-h752t\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:06.459745 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.459685    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-token\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:06.459745 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.459695    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-envoy\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:06.459745 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.459703    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f2bc3de-95f5-4a48-b9bd-1647907db60f-istio-data\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:06.904875 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.904838    2581 generic.go:356] "Generic (PLEG): container finished" podID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerID="9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7" exitCode=0
Jul 24 15:20:06.905309 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.904886    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"
Jul 24 15:20:06.905309 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.904920    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" event={"ID":"3f2bc3de-95f5-4a48-b9bd-1647907db60f","Type":"ContainerDied","Data":"9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7"}
Jul 24 15:20:06.905309 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.904973    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9" event={"ID":"3f2bc3de-95f5-4a48-b9bd-1647907db60f","Type":"ContainerDied","Data":"5970cdca04c1066d0f351947dd37d80915128b7eb4dd3e43eece3895a3ef335b"}
Jul 24 15:20:06.905309 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.904996    2581 scope.go:117] "RemoveContainer" containerID="9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7"
Jul 24 15:20:06.914990 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.914973    2581 scope.go:117] "RemoveContainer" containerID="7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14"
Jul 24 15:20:06.923623 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.923604    2581 scope.go:117] "RemoveContainer" containerID="9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7"
Jul 24 15:20:06.923930 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:20:06.923904    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7\": container with ID starting with 9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7 not found: ID does not exist" containerID="9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7"
Jul 24 15:20:06.923994 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.923944    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7"} err="failed to get container status \"9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7\": rpc error: code = NotFound desc = could not find container \"9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7\": container with ID starting with 9607e5d9415f950c2e08676215b3f64bb038523ddc13a37fdacb6eede9d312d7 not found: ID does not exist"
Jul 24 15:20:06.923994 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.923971    2581 scope.go:117] "RemoveContainer" containerID="7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14"
Jul 24 15:20:06.924257 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:20:06.924240    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14\": container with ID starting with 7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14 not found: ID does not exist" containerID="7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14"
Jul 24 15:20:06.924312 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.924266    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14"} err="failed to get container status \"7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14\": rpc error: code = NotFound desc = could not find container \"7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14\": container with ID starting with 7a3ac0f54315df11bc01698e15983689e3d411908fd66e94791bf09a2a718b14 not found: ID does not exist"
Jul 24 15:20:06.930751 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.930723    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"]
Jul 24 15:20:06.933844 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:06.933817    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2edd1c-openshift-default-d9cd7c4fd-znqt9"]
Jul 24 15:20:07.044107 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:07.044074    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"
Jul 24 15:20:07.165706 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:07.165606    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17704ad-e506-490f-9cb7-17e515c36e03-maas-api-service-ca\") pod \"d17704ad-e506-490f-9cb7-17e515c36e03\" (UID: \"d17704ad-e506-490f-9cb7-17e515c36e03\") "
Jul 24 15:20:07.165976 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:07.165953    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d17704ad-e506-490f-9cb7-17e515c36e03-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d17704ad-e506-490f-9cb7-17e515c36e03" (UID: "d17704ad-e506-490f-9cb7-17e515c36e03"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:20:07.266760 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:07.266722    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17704ad-e506-490f-9cb7-17e515c36e03-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:07.335111 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:07.335074    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" path="/var/lib/kubelet/pods/3f2bc3de-95f5-4a48-b9bd-1647907db60f/volumes"
Jul 24 15:20:07.911065 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:07.911038    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"
Jul 24 15:20:07.911065 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:07.911054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t" event={"ID":"d17704ad-e506-490f-9cb7-17e515c36e03","Type":"ContainerDied","Data":"24c6a0cf07bd3febbb1a1f60893767dc54838e4339a3bfd8d60dd6970540e50b"}
Jul 24 15:20:07.911563 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:07.911129    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24c6a0cf07bd3febbb1a1f60893767dc54838e4339a3bfd8d60dd6970540e50b"
Jul 24 15:20:21.206911 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:21.206875    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:20:24.519887 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:24.519847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:20:24.609526 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:24.609484    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:20:24.710585 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:24.710554    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:20:29.671938 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:29.671891    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:20:29.681526 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:29.681490    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"]
Jul 24 15:20:29.683855 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:29.683832    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-bb95eb-f247t"]
Jul 24 15:20:30.080429 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:30.080345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:20:31.335548 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:31.335517    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d17704ad-e506-490f-9cb7-17e515c36e03" path="/var/lib/kubelet/pods/d17704ad-e506-490f-9cb7-17e515c36e03/volumes"
Jul 24 15:20:35.049469 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:35.049435    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"]
Jul 24 15:20:35.106171 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:35.106134    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:20:35.520122 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:35.520074    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:20:50.069039 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.068981    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg" podUID="cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" containerName="main" containerID="cri-o://c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed" gracePeriod=45
Jul 24 15:20:50.322823 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.322756    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:20:50.454194 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.454146    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-home\") pod \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") "
Jul 24 15:20:50.454379 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.454224    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tmp-dir\") pod \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") "
Jul 24 15:20:50.454379 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.454257    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tls-certs\") pod \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") "
Jul 24 15:20:50.454379 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.454307    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6rn5\" (UniqueName: \"kubernetes.io/projected/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-kube-api-access-q6rn5\") pod \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") "
Jul 24 15:20:50.454379 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.454334    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-dshm\") pod \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") "
Jul 24 15:20:50.454611 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.454396    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-model-cache\") pod \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\" (UID: \"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6\") "
Jul 24 15:20:50.454611 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.454447    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" (UID: "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:20:50.454611 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.454464    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-home" (OuterVolumeSpecName: "home") pod "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" (UID: "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:20:50.454779 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.454665    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-home\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:50.454779 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.454681    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-model-cache" (OuterVolumeSpecName: "model-cache") pod "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" (UID: "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:20:50.454779 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.454689    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tmp-dir\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:50.456492 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.456465    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-dshm" (OuterVolumeSpecName: "dshm") pod "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" (UID: "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:20:50.456618 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.456563    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" (UID: "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:20:50.456618 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.456573    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-kube-api-access-q6rn5" (OuterVolumeSpecName: "kube-api-access-q6rn5") pod "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" (UID: "cdde6aef-9ab0-451b-a1a6-27b9e0b208a6"). InnerVolumeSpecName "kube-api-access-q6rn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:20:50.555144 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.555112    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-tls-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:50.555144 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.555138    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6rn5\" (UniqueName: \"kubernetes.io/projected/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-kube-api-access-q6rn5\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:50.555144 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.555148    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-dshm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:50.555370 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:50.555156    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:20:51.140063 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:51.140025    2581 generic.go:356] "Generic (PLEG): container finished" podID="cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" containerID="c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed" exitCode=0
Jul 24 15:20:51.140063 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:51.140066    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"
Jul 24 15:20:51.140637 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:51.140068    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg" event={"ID":"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6","Type":"ContainerDied","Data":"c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed"}
Jul 24 15:20:51.140637 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:51.140145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg" event={"ID":"cdde6aef-9ab0-451b-a1a6-27b9e0b208a6","Type":"ContainerDied","Data":"2c646b2e980b8ea5790976d4296e8b955bccfd0a03ebea7cb57599f3237972e2"}
Jul 24 15:20:51.140637 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:51.140171    2581 scope.go:117] "RemoveContainer" containerID="c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed"
Jul 24 15:20:51.149616 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:51.149599    2581 scope.go:117] "RemoveContainer" containerID="c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed"
Jul 24 15:20:51.149889 ip-10-0-143-246 kubenswrapper[2581]: E0724 15:20:51.149873    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed\": container with ID starting with c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed not found: ID does not exist" containerID="c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed"
Jul 24 15:20:51.149946 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:51.149897    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed"} err="failed to get container status \"c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed\": rpc error: code = NotFound desc = could not find container \"c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed\": container with ID starting with c841e9c6d93dba06d69b7f71c2d87e3aa838b48673eb81f2ab9507eea39aa6ed not found: ID does not exist"
Jul 24 15:20:51.161232 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:51.161207    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"]
Jul 24 15:20:51.164827 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:51.164808    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-bb95eb/ipp-route-bb95eb-kserve-557ff8d987-trdvg"]
Jul 24 15:20:51.334277 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:20:51.334242    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" path="/var/lib/kubelet/pods/cdde6aef-9ab0-451b-a1a6-27b9e0b208a6/volumes"
Jul 24 15:21:07.139530 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:07.139487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:07.238050 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:07.238011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:12.837504 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:12.837469    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:13.254588 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:13.254555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:19.016886 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:19.016849    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:24.922043 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:24.921940    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:25.023260 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.023208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:25.189161 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.189069    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"]
Jul 24 15:21:25.189400 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.189374    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy" containerID="cri-o://3c08bc3a8e024b043f580a0f1af7c0d3c7ad6e3c30827569a8b6cdd5f6182972" gracePeriod=30
Jul 24 15:21:25.207693 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.207667    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:25.436753 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.436722    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:25.553749 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.553669    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"]
Jul 24 15:21:25.554121 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.554087    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerName="istio-proxy"
Jul 24 15:21:25.554212 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.554122    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerName="istio-proxy"
Jul 24 15:21:25.554212 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.554137    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d17704ad-e506-490f-9cb7-17e515c36e03" containerName="revoke-keys"
Jul 24 15:21:25.554212 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.554144    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d17704ad-e506-490f-9cb7-17e515c36e03" containerName="revoke-keys"
Jul 24 15:21:25.554212 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.554179    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" containerName="main"
Jul 24 15:21:25.554212 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.554187    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" containerName="main"
Jul 24 15:21:25.554458 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.554296    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerName="istio-proxy"
Jul 24 15:21:25.554458 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.554309    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d17704ad-e506-490f-9cb7-17e515c36e03" containerName="revoke-keys"
Jul 24 15:21:25.554458 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.554319    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdde6aef-9ab0-451b-a1a6-27b9e0b208a6" containerName="main"
Jul 24 15:21:25.556743 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.556723    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"
Jul 24 15:21:25.559336 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.559315    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:21:25.559449 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.559411    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:21:25.566951 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.566926    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"]
Jul 24 15:21:25.681925 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.681888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b99466b-383d-4434-83d2-e10b9ff411ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c\" (UID: \"7b99466b-383d-4434-83d2-e10b9ff411ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"
Jul 24 15:21:25.782727 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.782683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b99466b-383d-4434-83d2-e10b9ff411ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c\" (UID: \"7b99466b-383d-4434-83d2-e10b9ff411ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"
Jul 24 15:21:25.783365 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.783347    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b99466b-383d-4434-83d2-e10b9ff411ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c\" (UID: \"7b99466b-383d-4434-83d2-e10b9ff411ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"
Jul 24 15:21:25.867905 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:25.867880    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"
Jul 24 15:21:26.026532 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:26.026506    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"]
Jul 24 15:21:26.028854 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:21:26.028828    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b99466b_383d_4434_83d2_e10b9ff411ac.slice/crio-442c4adc216830acf8255c825b62931bb8872570e6b3654375381f5d5623d598 WatchSource:0}: Error finding container 442c4adc216830acf8255c825b62931bb8872570e6b3654375381f5d5623d598: Status 404 returned error can't find the container with id 442c4adc216830acf8255c825b62931bb8872570e6b3654375381f5d5623d598
Jul 24 15:21:26.313103 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:26.313058    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c" event={"ID":"7b99466b-383d-4434-83d2-e10b9ff411ac","Type":"ContainerStarted","Data":"442c4adc216830acf8255c825b62931bb8872570e6b3654375381f5d5623d598"}
Jul 24 15:21:27.319984 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:27.319947    2581 generic.go:356] "Generic (PLEG): container finished" podID="7b99466b-383d-4434-83d2-e10b9ff411ac" containerID="e071a38cfac44c2a80807d27ae8be3db73829474be6c942a4f73509050f965a2" exitCode=0
Jul 24 15:21:27.320453 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:27.320034    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c" event={"ID":"7b99466b-383d-4434-83d2-e10b9ff411ac","Type":"ContainerDied","Data":"e071a38cfac44c2a80807d27ae8be3db73829474be6c942a4f73509050f965a2"}
Jul 24 15:21:28.453569 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:28.453546    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"
Jul 24 15:21:28.610370 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:28.610347    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b99466b-383d-4434-83d2-e10b9ff411ac-maas-api-service-ca\") pod \"7b99466b-383d-4434-83d2-e10b9ff411ac\" (UID: \"7b99466b-383d-4434-83d2-e10b9ff411ac\") "
Jul 24 15:21:28.610665 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:28.610644    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b99466b-383d-4434-83d2-e10b9ff411ac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b99466b-383d-4434-83d2-e10b9ff411ac" (UID: "7b99466b-383d-4434-83d2-e10b9ff411ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:21:28.711986 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:28.711956    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b99466b-383d-4434-83d2-e10b9ff411ac-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:21:29.332014 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:29.331989    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"
Jul 24 15:21:29.335364 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:29.335328    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c" event={"ID":"7b99466b-383d-4434-83d2-e10b9ff411ac","Type":"ContainerDied","Data":"442c4adc216830acf8255c825b62931bb8872570e6b3654375381f5d5623d598"}
Jul 24 15:21:29.335461 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:29.335392    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="442c4adc216830acf8255c825b62931bb8872570e6b3654375381f5d5623d598"
Jul 24 15:21:30.339601 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.339562    2581 generic.go:356] "Generic (PLEG): container finished" podID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerID="3c08bc3a8e024b043f580a0f1af7c0d3c7ad6e3c30827569a8b6cdd5f6182972" exitCode=0
Jul 24 15:21:30.340023 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.339630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" event={"ID":"6c7079eb-c2b0-4ad6-9296-66a4653c50d8","Type":"ContainerDied","Data":"3c08bc3a8e024b043f580a0f1af7c0d3c7ad6e3c30827569a8b6cdd5f6182972"}
Jul 24 15:21:30.340023 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.339685    2581 scope.go:117] "RemoveContainer" containerID="ff646259ceeda69328fd02a247b22b7e82cb5c6d83d22414feec641c935fe767"
Jul 24 15:21:30.437490 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.437463    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:21:30.632463 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.632434    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-podinfo\") pod \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") "
Jul 24 15:21:30.632656 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.632501    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8g9n\" (UniqueName: \"kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-kube-api-access-n8g9n\") pod \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") "
Jul 24 15:21:30.632656 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.632546    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-token\") pod \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") "
Jul 24 15:21:30.632656 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.632566    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-certs\") pod \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") "
Jul 24 15:21:30.632656 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.632595    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-credential-socket\") pod \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") "
Jul 24 15:21:30.632656 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.632624    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-socket\") pod \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") "
Jul 24 15:21:30.632656 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.632649    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-envoy\") pod \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") "
Jul 24 15:21:30.632990 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.632675    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-data\") pod \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") "
Jul 24 15:21:30.632990 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.632720    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istiod-ca-cert\") pod \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\" (UID: \"6c7079eb-c2b0-4ad6-9296-66a4653c50d8\") "
Jul 24 15:21:30.632990 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.632869    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6c7079eb-c2b0-4ad6-9296-66a4653c50d8" (UID: "6c7079eb-c2b0-4ad6-9296-66a4653c50d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:21:30.633162 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.633028    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:21:30.633162 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.633059    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6c7079eb-c2b0-4ad6-9296-66a4653c50d8" (UID: "6c7079eb-c2b0-4ad6-9296-66a4653c50d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:21:30.633272 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.633178    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6c7079eb-c2b0-4ad6-9296-66a4653c50d8" (UID: "6c7079eb-c2b0-4ad6-9296-66a4653c50d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:21:30.633321 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.633235    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6c7079eb-c2b0-4ad6-9296-66a4653c50d8" (UID: "6c7079eb-c2b0-4ad6-9296-66a4653c50d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:21:30.633962 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.633932    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-data" (OuterVolumeSpecName: "istio-data") pod "6c7079eb-c2b0-4ad6-9296-66a4653c50d8" (UID: "6c7079eb-c2b0-4ad6-9296-66a4653c50d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:21:30.634757 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.634734    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6c7079eb-c2b0-4ad6-9296-66a4653c50d8" (UID: "6c7079eb-c2b0-4ad6-9296-66a4653c50d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:21:30.634861 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.634816    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-kube-api-access-n8g9n" (OuterVolumeSpecName: "kube-api-access-n8g9n") pod "6c7079eb-c2b0-4ad6-9296-66a4653c50d8" (UID: "6c7079eb-c2b0-4ad6-9296-66a4653c50d8"). InnerVolumeSpecName "kube-api-access-n8g9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:21:30.634909 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.634878    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-token" (OuterVolumeSpecName: "istio-token") pod "6c7079eb-c2b0-4ad6-9296-66a4653c50d8" (UID: "6c7079eb-c2b0-4ad6-9296-66a4653c50d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:21:30.635023 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.635006    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6c7079eb-c2b0-4ad6-9296-66a4653c50d8" (UID: "6c7079eb-c2b0-4ad6-9296-66a4653c50d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:21:30.733844 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.733810    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-envoy\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:21:30.733844 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.733835    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-data\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:21:30.733844 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.733847    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istiod-ca-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:21:30.734171 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.733857    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-podinfo\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:21:30.734171 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.733869    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8g9n\" (UniqueName: \"kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-kube-api-access-n8g9n\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:21:30.734171 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.733882    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-istio-token\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:21:30.734171 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.733894    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-credential-socket\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:21:30.734171 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:30.733907    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c7079eb-c2b0-4ad6-9296-66a4653c50d8-workload-socket\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:21:31.345695 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:31.345669    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"
Jul 24 15:21:31.346116 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:31.345699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m" event={"ID":"6c7079eb-c2b0-4ad6-9296-66a4653c50d8","Type":"ContainerDied","Data":"191f1715a97e90021bd6fdfc2b016593ac712c63778e3fce56f6f1c001c2ead0"}
Jul 24 15:21:31.346116 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:31.345762    2581 scope.go:117] "RemoveContainer" containerID="3c08bc3a8e024b043f580a0f1af7c0d3c7ad6e3c30827569a8b6cdd5f6182972"
Jul 24 15:21:31.363736 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:31.363713    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"]
Jul 24 15:21:31.367696 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:31.367669    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-bb95eb-openshift-default-85cfd7766d-hmg6m"]
Jul 24 15:21:33.334653 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:33.334619    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" path="/var/lib/kubelet/pods/6c7079eb-c2b0-4ad6-9296-66a4653c50d8/volumes"
Jul 24 15:21:37.779742 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:37.779708    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:37.882748 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:37.882711    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:37.980935 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:37.980904    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:40.647361 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:40.647324    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"]
Jul 24 15:21:40.649975 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:40.649950    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ba0783-2k99c"]
Jul 24 15:21:40.651380 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:40.651357    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:41.047663 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:41.047576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:41.333823 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:41.333745    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b99466b-383d-4434-83d2-e10b9ff411ac" path="/var/lib/kubelet/pods/7b99466b-383d-4434-83d2-e10b9ff411ac/volumes"
Jul 24 15:21:41.714303 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:41.714270    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:21:47.584491 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:47.584453    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:47.689112 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:47.689060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:47.873462 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:47.873431    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:48.069929 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.069896    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:48.204588 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.204508    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"]
Jul 24 15:21:48.204891 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.204879    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b99466b-383d-4434-83d2-e10b9ff411ac" containerName="revoke-keys"
Jul 24 15:21:48.204933 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.204894    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b99466b-383d-4434-83d2-e10b9ff411ac" containerName="revoke-keys"
Jul 24 15:21:48.204933 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.204916    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy"
Jul 24 15:21:48.204933 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.204925    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy"
Jul 24 15:21:48.205031 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.204934    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy"
Jul 24 15:21:48.205031 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.204941    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy"
Jul 24 15:21:48.205031 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.204954    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerName="istio-proxy"
Jul 24 15:21:48.205031 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.204960    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerName="istio-proxy"
Jul 24 15:21:48.205031 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.205017    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy"
Jul 24 15:21:48.205031 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.205027    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c7079eb-c2b0-4ad6-9296-66a4653c50d8" containerName="istio-proxy"
Jul 24 15:21:48.205254 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.205034    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b99466b-383d-4434-83d2-e10b9ff411ac" containerName="revoke-keys"
Jul 24 15:21:48.205254 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.205041    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f2bc3de-95f5-4a48-b9bd-1647907db60f" containerName="istio-proxy"
Jul 24 15:21:48.208054 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.208038    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"
Jul 24 15:21:48.210527 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.210498    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:21:48.210527 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.210499    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:21:48.215825 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.215561    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"]
Jul 24 15:21:48.284202 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.284173    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3771d5f3-62e2-497a-8d0e-733b6ca0c2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl\" (UID: \"3771d5f3-62e2-497a-8d0e-733b6ca0c2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"
Jul 24 15:21:48.384892 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.384863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3771d5f3-62e2-497a-8d0e-733b6ca0c2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl\" (UID: \"3771d5f3-62e2-497a-8d0e-733b6ca0c2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"
Jul 24 15:21:48.385492 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.385473    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3771d5f3-62e2-497a-8d0e-733b6ca0c2db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl\" (UID: \"3771d5f3-62e2-497a-8d0e-733b6ca0c2db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"
Jul 24 15:21:48.519743 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.519664    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"
Jul 24 15:21:48.652190 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:48.652162    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"]
Jul 24 15:21:48.653963 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:21:48.653940    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3771d5f3_62e2_497a_8d0e_733b6ca0c2db.slice/crio-1a38359f9aeb1f64b812ccb29e5d033e54b8926da4be4566072bb3ebef70c6a3 WatchSource:0}: Error finding container 1a38359f9aeb1f64b812ccb29e5d033e54b8926da4be4566072bb3ebef70c6a3: Status 404 returned error can't find the container with id 1a38359f9aeb1f64b812ccb29e5d033e54b8926da4be4566072bb3ebef70c6a3
Jul 24 15:21:49.446780 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:49.446745    2581 generic.go:356] "Generic (PLEG): container finished" podID="3771d5f3-62e2-497a-8d0e-733b6ca0c2db" containerID="075a5e60d627806a2646c14fc1f9fdb3a6be10bcda96e2e757f556b73da504b3" exitCode=0
Jul 24 15:21:49.446949 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:49.446826    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl" event={"ID":"3771d5f3-62e2-497a-8d0e-733b6ca0c2db","Type":"ContainerDied","Data":"075a5e60d627806a2646c14fc1f9fdb3a6be10bcda96e2e757f556b73da504b3"}
Jul 24 15:21:49.446949 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:49.446878    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl" event={"ID":"3771d5f3-62e2-497a-8d0e-733b6ca0c2db","Type":"ContainerStarted","Data":"1a38359f9aeb1f64b812ccb29e5d033e54b8926da4be4566072bb3ebef70c6a3"}
Jul 24 15:21:50.580705 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:50.580682    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"
Jul 24 15:21:50.709551 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:50.709435    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3771d5f3-62e2-497a-8d0e-733b6ca0c2db-maas-api-service-ca\") pod \"3771d5f3-62e2-497a-8d0e-733b6ca0c2db\" (UID: \"3771d5f3-62e2-497a-8d0e-733b6ca0c2db\") "
Jul 24 15:21:50.709807 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:50.709785    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3771d5f3-62e2-497a-8d0e-733b6ca0c2db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3771d5f3-62e2-497a-8d0e-733b6ca0c2db" (UID: "3771d5f3-62e2-497a-8d0e-733b6ca0c2db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:21:50.810528 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:50.810499    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3771d5f3-62e2-497a-8d0e-733b6ca0c2db-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:21:51.459577 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:51.459546    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"
Jul 24 15:21:51.459577 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:51.459568    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl" event={"ID":"3771d5f3-62e2-497a-8d0e-733b6ca0c2db","Type":"ContainerDied","Data":"1a38359f9aeb1f64b812ccb29e5d033e54b8926da4be4566072bb3ebef70c6a3"}
Jul 24 15:21:51.459813 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:51.459621    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a38359f9aeb1f64b812ccb29e5d033e54b8926da4be4566072bb3ebef70c6a3"
Jul 24 15:21:58.730470 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:58.730434    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5"]
Jul 24 15:21:58.734126 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:58.734086    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dc7d81-29748435-x7hp5"]
Jul 24 15:21:58.734770 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:58.734745    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:58.809923 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:58.809892    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:58.911737 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:58.911701    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:21:59.334464 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:21:59.334432    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="892d9632-478f-4532-9f37-f6d7749fec8d" path="/var/lib/kubelet/pods/892d9632-478f-4532-9f37-f6d7749fec8d/volumes"
Jul 24 15:22:01.903573 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:01.903542    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:22:03.283667 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:03.283632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:03.288062 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:03.288034    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"]
Jul 24 15:22:03.292746 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:03.292716    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dc7d81-9gkhl"]
Jul 24 15:22:03.334425 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:03.334398    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3771d5f3-62e2-497a-8d0e-733b6ca0c2db" path="/var/lib/kubelet/pods/3771d5f3-62e2-497a-8d0e-733b6ca0c2db/volumes"
Jul 24 15:22:03.680396 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:03.680362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:03.722473 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:03.722441    2581 scope.go:117] "RemoveContainer" containerID="5b432bcd1420d430266977740f3ef9d5c4c3fb9ea9644792d0451dc2a796cebb"
Jul 24 15:22:09.917819 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:09.917777    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:09.986248 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:09.986214    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:10.195850 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.195758    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:10.428015 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.427978    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:10.527341 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.527257    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"]
Jul 24 15:22:10.527834 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.527814    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3771d5f3-62e2-497a-8d0e-733b6ca0c2db" containerName="revoke-keys"
Jul 24 15:22:10.527913 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.527835    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3771d5f3-62e2-497a-8d0e-733b6ca0c2db" containerName="revoke-keys"
Jul 24 15:22:10.527983 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.527923    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3771d5f3-62e2-497a-8d0e-733b6ca0c2db" containerName="revoke-keys"
Jul 24 15:22:10.533036 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.532841    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"
Jul 24 15:22:10.535650 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.535565    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:22:10.536934 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.536807    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8zr8n\""
Jul 24 15:22:10.539230 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.539208    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"]
Jul 24 15:22:10.693187 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.693147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ada3323d-9d63-440b-81fe-fe5dfd5eaa10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q\" (UID: \"ada3323d-9d63-440b-81fe-fe5dfd5eaa10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"
Jul 24 15:22:10.794064 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.793964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ada3323d-9d63-440b-81fe-fe5dfd5eaa10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q\" (UID: \"ada3323d-9d63-440b-81fe-fe5dfd5eaa10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"
Jul 24 15:22:10.794639 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.794614    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ada3323d-9d63-440b-81fe-fe5dfd5eaa10-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q\" (UID: \"ada3323d-9d63-440b-81fe-fe5dfd5eaa10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"
Jul 24 15:22:10.844723 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.844699    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"
Jul 24 15:22:10.978243 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:10.978220    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"]
Jul 24 15:22:10.980002 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:22:10.979974    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada3323d_9d63_440b_81fe_fe5dfd5eaa10.slice/crio-fb9d40acbfc4804a3d5b2af7f6cb0ff7f8fea40de0b0f9266c9353e7d0aadc08 WatchSource:0}: Error finding container fb9d40acbfc4804a3d5b2af7f6cb0ff7f8fea40de0b0f9266c9353e7d0aadc08: Status 404 returned error can't find the container with id fb9d40acbfc4804a3d5b2af7f6cb0ff7f8fea40de0b0f9266c9353e7d0aadc08
Jul 24 15:22:11.567217 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:11.567132    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q" event={"ID":"ada3323d-9d63-440b-81fe-fe5dfd5eaa10","Type":"ContainerStarted","Data":"799135275b059e956eaf913fbace02a5772c161b6b16d1d8b1388efdd62061a0"}
Jul 24 15:22:11.567217 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:11.567189    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q" event={"ID":"ada3323d-9d63-440b-81fe-fe5dfd5eaa10","Type":"ContainerStarted","Data":"fb9d40acbfc4804a3d5b2af7f6cb0ff7f8fea40de0b0f9266c9353e7d0aadc08"}
Jul 24 15:22:11.594473 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:11.594428    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q" podStartSLOduration=1.5944131160000001 podStartE2EDuration="1.594413116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:22:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:22:11.592828439 +0000 UTC m=+3010.096546408" watchObservedRunningTime="2026-07-24 15:22:11.594413116 +0000 UTC m=+3010.098131073"
Jul 24 15:22:11.783537 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:11.783506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:12.573482 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:12.573451    2581 generic.go:356] "Generic (PLEG): container finished" podID="ada3323d-9d63-440b-81fe-fe5dfd5eaa10" containerID="799135275b059e956eaf913fbace02a5772c161b6b16d1d8b1388efdd62061a0" exitCode=0
Jul 24 15:22:12.573858 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:12.573489    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q" event={"ID":"ada3323d-9d63-440b-81fe-fe5dfd5eaa10","Type":"ContainerDied","Data":"799135275b059e956eaf913fbace02a5772c161b6b16d1d8b1388efdd62061a0"}
Jul 24 15:22:13.706085 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:13.706062    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"
Jul 24 15:22:13.824362 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:13.824332    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ada3323d-9d63-440b-81fe-fe5dfd5eaa10-maas-api-service-ca\") pod \"ada3323d-9d63-440b-81fe-fe5dfd5eaa10\" (UID: \"ada3323d-9d63-440b-81fe-fe5dfd5eaa10\") "
Jul 24 15:22:13.824695 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:13.824673    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ada3323d-9d63-440b-81fe-fe5dfd5eaa10-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ada3323d-9d63-440b-81fe-fe5dfd5eaa10" (UID: "ada3323d-9d63-440b-81fe-fe5dfd5eaa10"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:22:13.925809 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:13.925782    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ada3323d-9d63-440b-81fe-fe5dfd5eaa10-maas-api-service-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Jul 24 15:22:14.584014 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:14.583972    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q" event={"ID":"ada3323d-9d63-440b-81fe-fe5dfd5eaa10","Type":"ContainerDied","Data":"fb9d40acbfc4804a3d5b2af7f6cb0ff7f8fea40de0b0f9266c9353e7d0aadc08"}
Jul 24 15:22:14.584204 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:14.584028    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb9d40acbfc4804a3d5b2af7f6cb0ff7f8fea40de0b0f9266c9353e7d0aadc08"
Jul 24 15:22:14.584204 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:14.583995    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"
Jul 24 15:22:20.978195 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:20.978162    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4"]
Jul 24 15:22:20.983385 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:20.983361    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8ea489-29748435-pndz4"]
Jul 24 15:22:20.987380 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:20.987349    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:21.087878 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:21.087841    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:21.143447 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:21.143413    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:21.335202 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:21.335130    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9edd38ee-c750-401d-b646-f106c2cd5081" path="/var/lib/kubelet/pods/9edd38ee-c750-401d-b646-f106c2cd5081/volumes"
Jul 24 15:22:24.004057 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:24.004019    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5jwnw"]
Jul 24 15:22:25.610353 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:25.610283    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"]
Jul 24 15:22:25.610817 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:25.610382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:25.612832 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:25.612811    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8ea489-bvr7q"]
Jul 24 15:22:25.970623 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:25.970524    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:27.334539 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:27.334500    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ada3323d-9d63-440b-81fe-fe5dfd5eaa10" path="/var/lib/kubelet/pods/ada3323d-9d63-440b-81fe-fe5dfd5eaa10/volumes"
Jul 24 15:22:31.855789 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:31.855754    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:31.921606 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:31.921539    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:32.158602 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:32.158571    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:32.353019 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:32.352980    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:38.092710 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:38.092675    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wxdd4_e2ea64ef-1529-4ef7-a910-17a76d0bb6b3/manager/0.log"
Jul 24 15:22:38.217817 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:38.217785    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hk8t8_a89d208d-ce0f-4a37-9472-a194a89a111d/manager/0.log"
Jul 24 15:22:38.701432 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:38.701406    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp_4b7741a8-9b14-4135-95fe-bf60afa9bc74/copy-manifests/0.log"
Jul 24 15:22:38.708376 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:38.708347    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp_4b7741a8-9b14-4135-95fe-bf60afa9bc74/manager/0.log"
Jul 24 15:22:39.531370 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:39.531346    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748420-wtbxj_fc0cc1ba-2891-4a71-910c-bbbc89fc1243/cleanup/0.log"
Jul 24 15:22:39.640133 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:39.640106    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748435-hdn7x_b12d1f2f-ee6a-4d78-96c1-0f60bd19ba0f/cleanup/0.log"
Jul 24 15:22:41.744395 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:41.744367    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5jwnw_745928f7-2edc-42d3-8be0-d3cbe0ba5988/limitador/0.log"
Jul 24 15:22:42.561606 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:42.561558    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55ffdfd896-w6h5l_a70f0840-aeb0-4b5e-b183-df0046491fbf/kube-auth-proxy/0.log"
Jul 24 15:22:43.553439 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:43.553408    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5hh4f_477a40a4-df3e-4fb0-8338-ddb4acb7274d/main/0.log"
Jul 24 15:22:44.034803 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:44.034769    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-rpw9v_dcc9a946-328e-4497-b7e0-620008c93c73/main/0.log"
Jul 24 15:22:49.467111 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.467064    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-84bxh/must-gather-g56hx"]
Jul 24 15:22:49.467628 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.467609    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ada3323d-9d63-440b-81fe-fe5dfd5eaa10" containerName="revoke-keys"
Jul 24 15:22:49.467711 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.467632    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ada3323d-9d63-440b-81fe-fe5dfd5eaa10" containerName="revoke-keys"
Jul 24 15:22:49.467776 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.467713    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ada3323d-9d63-440b-81fe-fe5dfd5eaa10" containerName="revoke-keys"
Jul 24 15:22:49.470870 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.470851    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84bxh/must-gather-g56hx"
Jul 24 15:22:49.473571 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.473551    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-84bxh\"/\"openshift-service-ca.crt\""
Jul 24 15:22:49.474134 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.474113    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-84bxh\"/\"default-dockercfg-48m4q\""
Jul 24 15:22:49.474756 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.474742    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-84bxh\"/\"kube-root-ca.crt\""
Jul 24 15:22:49.484009 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.483985    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84bxh/must-gather-g56hx"]
Jul 24 15:22:49.493524 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.493497    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:49.559262 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.559228    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqk5n\" (UniqueName: \"kubernetes.io/projected/79449e1c-cdd7-4f46-9583-7e5d644037e6-kube-api-access-wqk5n\") pod \"must-gather-g56hx\" (UID: \"79449e1c-cdd7-4f46-9583-7e5d644037e6\") " pod="openshift-must-gather-84bxh/must-gather-g56hx"
Jul 24 15:22:49.559262 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.559266    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/79449e1c-cdd7-4f46-9583-7e5d644037e6-must-gather-output\") pod \"must-gather-g56hx\" (UID: \"79449e1c-cdd7-4f46-9583-7e5d644037e6\") " pod="openshift-must-gather-84bxh/must-gather-g56hx"
Jul 24 15:22:49.660809 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.660771    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqk5n\" (UniqueName: \"kubernetes.io/projected/79449e1c-cdd7-4f46-9583-7e5d644037e6-kube-api-access-wqk5n\") pod \"must-gather-g56hx\" (UID: \"79449e1c-cdd7-4f46-9583-7e5d644037e6\") " pod="openshift-must-gather-84bxh/must-gather-g56hx"
Jul 24 15:22:49.660809 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.660810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/79449e1c-cdd7-4f46-9583-7e5d644037e6-must-gather-output\") pod \"must-gather-g56hx\" (UID: \"79449e1c-cdd7-4f46-9583-7e5d644037e6\") " pod="openshift-must-gather-84bxh/must-gather-g56hx"
Jul 24 15:22:49.661117 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.661079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/79449e1c-cdd7-4f46-9583-7e5d644037e6-must-gather-output\") pod \"must-gather-g56hx\" (UID: \"79449e1c-cdd7-4f46-9583-7e5d644037e6\") " pod="openshift-must-gather-84bxh/must-gather-g56hx"
Jul 24 15:22:49.671355 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.671333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqk5n\" (UniqueName: \"kubernetes.io/projected/79449e1c-cdd7-4f46-9583-7e5d644037e6-kube-api-access-wqk5n\") pod \"must-gather-g56hx\" (UID: \"79449e1c-cdd7-4f46-9583-7e5d644037e6\") " pod="openshift-must-gather-84bxh/must-gather-g56hx"
Jul 24 15:22:49.780480 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.780403    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84bxh/must-gather-g56hx"
Jul 24 15:22:49.920814 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.920786    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84bxh/must-gather-g56hx"]
Jul 24 15:22:49.923104 ip-10-0-143-246 kubenswrapper[2581]: W0724 15:22:49.923063    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79449e1c_cdd7_4f46_9583_7e5d644037e6.slice/crio-a45382aaec8e00d1c96d01bf91dec998123a312bf199fb7edbec0b5f1b453d1c WatchSource:0}: Error finding container a45382aaec8e00d1c96d01bf91dec998123a312bf199fb7edbec0b5f1b453d1c: Status 404 returned error can't find the container with id a45382aaec8e00d1c96d01bf91dec998123a312bf199fb7edbec0b5f1b453d1c
Jul 24 15:22:49.924993 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:49.924976    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:22:50.731366 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:50.731331    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log" limit=1024
Jul 24 15:22:50.759364 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:50.759336    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84bxh/must-gather-g56hx" event={"ID":"79449e1c-cdd7-4f46-9583-7e5d644037e6","Type":"ContainerStarted","Data":"a45382aaec8e00d1c96d01bf91dec998123a312bf199fb7edbec0b5f1b453d1c"}
Jul 24 15:22:51.768322 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:51.768281    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84bxh/must-gather-g56hx" event={"ID":"79449e1c-cdd7-4f46-9583-7e5d644037e6","Type":"ContainerStarted","Data":"7882c12edcf04479c2c2a85771b3e97fc77caf6c313f2fc2dff44f1f6a2b2a8f"}
Jul 24 15:22:51.768798 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:51.768335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84bxh/must-gather-g56hx" event={"ID":"79449e1c-cdd7-4f46-9583-7e5d644037e6","Type":"ContainerStarted","Data":"0ea662c4e2c0667d316ba8170d1e0ccbd4742a715db088bf84f76832e9ce984a"}
Jul 24 15:22:51.787731 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:51.787680    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-84bxh/must-gather-g56hx" podStartSLOduration=1.935071937 podStartE2EDuration="2.787659446s" totalImagesPullingTime="852.587509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:22:51.784114678 +0000 UTC m=+3050.287832643" watchObservedRunningTime="2026-07-24 15:22:51.787659446 +0000 UTC m=+3050.291377437"
Jul 24 15:22:52.390566 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:52.390532    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-snn2w_8f983040-8d3c-4f7c-8c82-12586a4033ee/global-pull-secret-syncer/0.log"
Jul 24 15:22:52.496025 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:52.495976    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g44tl_7f98367c-0a6f-491a-a185-8f34de518278/konnectivity-agent/0.log"
Jul 24 15:22:52.655555 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:52.655470    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-246.ec2.internal_346748fe11d636dec2275399a789dab0/haproxy/0.log"
Jul 24 15:22:57.365530 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:57.365499    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5jwnw_745928f7-2edc-42d3-8be0-d3cbe0ba5988/limitador/0.log"
Jul 24 15:22:59.110039 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.110000    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51d07f2b-dec7-485f-a725-93d754508b1c/alertmanager/0.log"
Jul 24 15:22:59.134142 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.134085    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51d07f2b-dec7-485f-a725-93d754508b1c/config-reloader/0.log"
Jul 24 15:22:59.159543 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.159511    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51d07f2b-dec7-485f-a725-93d754508b1c/kube-rbac-proxy-web/0.log"
Jul 24 15:22:59.184327 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.184292    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51d07f2b-dec7-485f-a725-93d754508b1c/kube-rbac-proxy/0.log"
Jul 24 15:22:59.208553 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.208521    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51d07f2b-dec7-485f-a725-93d754508b1c/kube-rbac-proxy-metric/0.log"
Jul 24 15:22:59.233675 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.233642    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51d07f2b-dec7-485f-a725-93d754508b1c/prom-label-proxy/0.log"
Jul 24 15:22:59.258617 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.258588    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_51d07f2b-dec7-485f-a725-93d754508b1c/init-config-reloader/0.log"
Jul 24 15:22:59.447339 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.447311    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-xdz88_eaf81ce3-20cc-411a-b776-09d5d286e4c5/monitoring-plugin/0.log"
Jul 24 15:22:59.647176 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.647072    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xbksj_f652c78e-d83f-40d7-9211-579f2e44c46b/node-exporter/0.log"
Jul 24 15:22:59.671568 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.671536    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xbksj_f652c78e-d83f-40d7-9211-579f2e44c46b/kube-rbac-proxy/0.log"
Jul 24 15:22:59.694236 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.694211    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xbksj_f652c78e-d83f-40d7-9211-579f2e44c46b/init-textfile/0.log"
Jul 24 15:22:59.839774 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.839750    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b862ffde-981d-4e89-8708-440f247e3a9b/prometheus/0.log"
Jul 24 15:22:59.867179 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.867147    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b862ffde-981d-4e89-8708-440f247e3a9b/config-reloader/0.log"
Jul 24 15:22:59.904498 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.903236    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b862ffde-981d-4e89-8708-440f247e3a9b/thanos-sidecar/0.log"
Jul 24 15:22:59.934681 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.934653    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b862ffde-981d-4e89-8708-440f247e3a9b/kube-rbac-proxy-web/0.log"
Jul 24 15:22:59.962808 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.962770    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b862ffde-981d-4e89-8708-440f247e3a9b/kube-rbac-proxy/0.log"
Jul 24 15:22:59.994260 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:22:59.994231    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b862ffde-981d-4e89-8708-440f247e3a9b/kube-rbac-proxy-thanos/0.log"
Jul 24 15:23:00.018674 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.018644    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b862ffde-981d-4e89-8708-440f247e3a9b/init-config-reloader/0.log"
Jul 24 15:23:00.052575 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.052542    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-xhlxv_e3d13857-8b7f-4996-9041-5fd1f8f84111/prometheus-operator/0.log"
Jul 24 15:23:00.073678 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.073650    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-xhlxv_e3d13857-8b7f-4996-9041-5fd1f8f84111/kube-rbac-proxy/0.log"
Jul 24 15:23:00.102953 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.102920    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-94nkg_dbf39a86-7562-46ec-97a7-2b01531d9b2d/prometheus-operator-admission-webhook/0.log"
Jul 24 15:23:00.137361 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.137328    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-787bc58487-lhtx8_d924890d-3b15-4fc5-a1b2-715405d2f28c/telemeter-client/0.log"
Jul 24 15:23:00.164603 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.164526    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-787bc58487-lhtx8_d924890d-3b15-4fc5-a1b2-715405d2f28c/reload/0.log"
Jul 24 15:23:00.188439 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.188408    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-787bc58487-lhtx8_d924890d-3b15-4fc5-a1b2-715405d2f28c/kube-rbac-proxy/0.log"
Jul 24 15:23:00.223761 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.223731    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc45f789-8mtnb_4590de7e-2a93-4e19-98a4-eb7d3cc27c5f/thanos-query/0.log"
Jul 24 15:23:00.249650 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.249620    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc45f789-8mtnb_4590de7e-2a93-4e19-98a4-eb7d3cc27c5f/kube-rbac-proxy-web/0.log"
Jul 24 15:23:00.276655 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.276623    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc45f789-8mtnb_4590de7e-2a93-4e19-98a4-eb7d3cc27c5f/kube-rbac-proxy/0.log"
Jul 24 15:23:00.310275 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.310208    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc45f789-8mtnb_4590de7e-2a93-4e19-98a4-eb7d3cc27c5f/prom-label-proxy/0.log"
Jul 24 15:23:00.341441 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.341413    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc45f789-8mtnb_4590de7e-2a93-4e19-98a4-eb7d3cc27c5f/kube-rbac-proxy-rules/0.log"
Jul 24 15:23:00.364565 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:00.364534    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66bc45f789-8mtnb_4590de7e-2a93-4e19-98a4-eb7d3cc27c5f/kube-rbac-proxy-metrics/0.log"
Jul 24 15:23:02.995582 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:02.995540    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"]
Jul 24 15:23:03.000538 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.000510    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.009743 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.009713    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"]
Jul 24 15:23:03.020528 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.020494    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-sys\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.020704 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.020537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-proc\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.020704 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.020611    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-lib-modules\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.020704 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.020642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-podres\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.020704 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.020673    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw6fs\" (UniqueName: \"kubernetes.io/projected/1580839d-44cc-452e-b670-a7ccc2425be8-kube-api-access-dw6fs\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.121609 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.121566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-sys\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.121809 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.121618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-proc\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.121809 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.121679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-lib-modules\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.121809 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.121706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-podres\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.121809 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.121708    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-sys\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.121809 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.121738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw6fs\" (UniqueName: \"kubernetes.io/projected/1580839d-44cc-452e-b670-a7ccc2425be8-kube-api-access-dw6fs\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.122139 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.121857    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-lib-modules\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.122139 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.121906    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-proc\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.122139 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.121978    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1580839d-44cc-452e-b670-a7ccc2425be8-podres\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.131025 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.130995    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw6fs\" (UniqueName: \"kubernetes.io/projected/1580839d-44cc-452e-b670-a7ccc2425be8-kube-api-access-dw6fs\") pod \"perf-node-gather-daemonset-7hvb2\" (UID: \"1580839d-44cc-452e-b670-a7ccc2425be8\") " pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.315463 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.315381    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.496825 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.496799    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"]
Jul 24 15:23:03.748188 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.748156    2581 scope.go:117] "RemoveContainer" containerID="24a3b93264017f8be937a5773038813146ff7be510394973016fb51b0604d072"
Jul 24 15:23:03.870193 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.870149    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2" event={"ID":"1580839d-44cc-452e-b670-a7ccc2425be8","Type":"ContainerStarted","Data":"d80809ced6834bdf46f65d1893422ae7643e31ece92887a6ff245cbfea5065de"}
Jul 24 15:23:03.870854 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.870831    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:03.870987 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.870955    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2" event={"ID":"1580839d-44cc-452e-b670-a7ccc2425be8","Type":"ContainerStarted","Data":"b1e355cfe5449872ad2b756cc146847c194bec973fbf51f88866b9ec66fe2a52"}
Jul 24 15:23:03.888837 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:03.888769    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2" podStartSLOduration=1.888745175 podStartE2EDuration="1.888745175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:23:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:23:03.886355732 +0000 UTC m=+3062.390073695" watchObservedRunningTime="2026-07-24 15:23:03.888745175 +0000 UTC m=+3062.392463134"
Jul 24 15:23:04.919054 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:04.919025    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fxnnt_80e703d4-7f37-4591-9528-1a53effbe5c7/dns/0.log"
Jul 24 15:23:04.939521 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:04.939494    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fxnnt_80e703d4-7f37-4591-9528-1a53effbe5c7/kube-rbac-proxy/0.log"
Jul 24 15:23:05.052233 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:05.052207    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fplzf_7500633c-648e-4a58-bb81-54bd2d438612/dns-node-resolver/0.log"
Jul 24 15:23:05.687941 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:05.687910    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fmzbh_0d372f67-c893-46a9-8ce9-64507e3ae5ea/node-ca/0.log"
Jul 24 15:23:06.978165 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:06.978136    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55ffdfd896-w6h5l_a70f0840-aeb0-4b5e-b183-df0046491fbf/kube-auth-proxy/0.log"
Jul 24 15:23:07.963175 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:07.963145    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cq9pm_5e6f93bf-25de-425e-b119-96cbf90b4075/serve-healthcheck-canary/0.log"
Jul 24 15:23:08.653903 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:08.653874    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r5znj_d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f/kube-rbac-proxy/0.log"
Jul 24 15:23:08.690483 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:08.690451    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r5znj_d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f/exporter/0.log"
Jul 24 15:23:08.727805 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:08.727769    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r5znj_d88021fe-bdc7-4953-8bb5-eeeb8bb33b8f/extractor/0.log"
Jul 24 15:23:10.897802 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:10.897775    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-84bxh/perf-node-gather-daemonset-7hvb2"
Jul 24 15:23:10.967645 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:10.967616    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wxdd4_e2ea64ef-1529-4ef7-a910-17a76d0bb6b3/manager/0.log"
Jul 24 15:23:11.366325 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:11.366280    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hk8t8_a89d208d-ce0f-4a37-9472-a194a89a111d/manager/0.log"
Jul 24 15:23:11.877361 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:11.877256    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp_4b7741a8-9b14-4135-95fe-bf60afa9bc74/manager/0.log"
Jul 24 15:23:11.905152 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:11.905122    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cffdd4dd6-2h7zp_4b7741a8-9b14-4135-95fe-bf60afa9bc74/copy-manifests/0.log"
Jul 24 15:23:13.467591 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:13.467560    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-55664b4b95-v9r5m_8a3a460e-7b62-4aee-b635-bfe181c84281/manager/0.log"
Jul 24 15:23:13.951463 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:13.951432    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55ffdfd896-w6h5l_a70f0840-aeb0-4b5e-b183-df0046491fbf/kube-auth-proxy/0.log"
Jul 24 15:23:15.791194 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:15.791163    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55ffdfd896-w6h5l_a70f0840-aeb0-4b5e-b183-df0046491fbf/kube-auth-proxy/0.log"
Jul 24 15:23:20.283188 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:20.283118    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wh679_c1aafd01-1d92-4f6e-b4d3-aceee2f33332/kube-multus-additional-cni-plugins/0.log"
Jul 24 15:23:20.305762 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:20.305734    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wh679_c1aafd01-1d92-4f6e-b4d3-aceee2f33332/egress-router-binary-copy/0.log"
Jul 24 15:23:20.328739 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:20.328716    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wh679_c1aafd01-1d92-4f6e-b4d3-aceee2f33332/cni-plugins/0.log"
Jul 24 15:23:20.351498 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:20.351475    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wh679_c1aafd01-1d92-4f6e-b4d3-aceee2f33332/bond-cni-plugin/0.log"
Jul 24 15:23:20.374245 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:20.374216    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wh679_c1aafd01-1d92-4f6e-b4d3-aceee2f33332/routeoverride-cni/0.log"
Jul 24 15:23:20.398560 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:20.398495    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wh679_c1aafd01-1d92-4f6e-b4d3-aceee2f33332/whereabouts-cni-bincopy/0.log"
Jul 24 15:23:20.422178 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:20.422152    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wh679_c1aafd01-1d92-4f6e-b4d3-aceee2f33332/whereabouts-cni/0.log"
Jul 24 15:23:20.463740 ip-10-0-143-246 kubenswrapper[2581]: I0724 15:23:20.463702    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nzj76_ade776d8-e297-4e5c-b57e-8e6cd942c159/kube-multus/0.log"